KITINet: Kinetics Theory Inspired Network Architectures with PDE Simulation Approaches
Abstract
Despite the widely recognized success of residual connections in modern neural networks, their design principles remain largely heuristic. This paper introduces KITINet (Kinetics Theory Inspired Network), a novel architecture that reinterprets feature propagation through the lens of non-equilibrium particle dynamics and partial differential equation (PDE) simulation. At its core, we propose a residual module that models feature updates as the stochastic evolution of a particle system, numerically simulated via a discretized solver for the Boltzmann transport equation (BTE). This formulation mimics particle collisions and energy exchange, enabling adaptive feature refinement via physics-informed interactions. Additionally, we reveal that this mechanism induces network parameter condensation during training, where parameters progressively concentrate into a sparse subset of dominant channels. Experiments on scientific computation (PDE operator), image classification (CIFAR-10/100), and text classification (IMDb/SNLI) show consistent improvements over classic network baselines, with negligible increase of FLOPs.
Cite
@article{arxiv.2505.17919,
title = {KITINet: Kinetics Theory Inspired Network Architectures with PDE Simulation Approaches},
author = {Mingquan Feng and Yifan Fu and Tongcheng Zhang and Yu Jiang and Yixin Huang and Junchi Yan},
journal= {arXiv preprint arXiv:2505.17919},
year = {2025}
}