Related papers: Homogenized Transformers
A driven system of three species of particle diffusing on a ring is studied in detail. The dynamics is local and conserves the three densities. A simple argument suggesting that the model should phase separate and break the translational…
We consider driven many-particle models which have a phase transition between an active and an absorbing phase. Like previously studied models, we have particle conservation, but here we introduce an additional symmetry - when two particles…
Discrete and continuous variables oftentimes require different treatments in many learning tasks. Identifying the Hamiltonian governing the evolution of a quantum system is a fundamental task in quantum learning theory. While previous works…
We investigate sedimentation of model hard sphere-like colloidal dispersions confined in horizontal capillaries using laser scanning confocal microscopy, dynamical density functional theory, and Brownian dynamics computer simulations. For…
The paper studies homogenization problem for a non-autonomous parabolic equation with a large random rapidly oscillating potential in the case of one dimensional spatial variable. We show that if the potential is a statistically homogeneous…
We study spectral form factor in periodically-kicked bosonic chains. We consider a family of models where a Hamiltonian with the terms diagonal in the Fock space basis, including random chemical potentials and pair-wise interactions, is…
The self-attention mechanism, while foundational to modern Transformer architectures, suffers from a critical inefficiency: it frequently allocates substantial attention to redundant or noisy context. Differential Attention addressed this…
We consider the mean field Fokker-Planck equation subject to nonlinear no-flux boundary conditions, which necessarily arise when subjecting a system of Brownian particles interacting via a pair potential in a bounded domain. With the…
A stochastic model for a mobile network is studied. Users enter the network, and then perform independent Markovian routes between nodes where they receive service according to the Processor-Sharing policy. Once their service requirement is…
This paper introduces a continuous-time stochastic dynamical framework for understanding how large language models (LLMs) may self-amplify latent biases or toxicity through their own chain-of-thought reasoning. The model posits an…
This work develops a dynamic homogenization approach for metamaterials. It finds an approximate macroscopic homogenized equation with constant coefficients posed in space and time; however, the resulting homogenized equation is higher order…
We investigate the hydrodynamic properties of a fluid simulated with a mesoscopic solvent model. Two distinct regimes are identified, the `particle regime' in which the dynamics is gas-like, and the `collective regime' where the dynamics is…
We study the continuum limit of discrete, nonconvex energy functionals defined on crystal lattices in dimensions $d\geq 2$. Since we are interested in energy functionals with random (stationary and ergodic) pair interactions, our problem…
This paper focuses on the long-term behavior of solutions to nonlinear stochastic Fokker-Planck equations driven by common noise, where the drift term has a linear dependence on the measure. These equations, which describe the evolution of…
The recently proposed physics-based framework by Huo and Johnson~\cite{huo2024capturing} models the attention mechanism of Large Language Models (LLMs) as an interacting two-body spin system, offering a first-principles explanation for…
The explicit construction of non-dispersive flat band modes and the tunability of has been reported for a hierarchical 3-simplex fractal geometry. A single band tight binding Hamiltonian defined for the deterministic self-similar…
Transformer-based models have achieved remarkable success across a wide range of domains, yet our understanding of their training dynamics remains limited. In this work, we identify a recurrent focus-dilution cycle in attention learning and…
We consider self-propelled rigid-bodies interacting through local body-attitude alignment modelled by stochastic differential equations. We derive a hydrodynamic model of this system at large spatio-temporal scales and particle numbers in…
We study a set of models of self-propelled particles that achieve collective motion through similar alignment-based dynamics, considering versions with and without repulsive interactions that do not affect the heading directions. We explore…
Confined granular fluids, placed in a shallow box that is vibrated vertically, can achieve homogeneous stationary states thanks to energy injection mechanisms that take place throughout the system. These states can be stable even at high…