Related papers: Homogenized Transformers
We develop a rigorous treatment of discontinuous stochastic unitary evolution for a system of quantum particles that interacts singularly with quantum "bubbles" at random instants of time. This model of a "cloud chamber" allows to watch and…
A stochastic treatment yielding to the derivation of a general Fokker-Planck equation is presented to model the slow convergence towards equilibrium of mean-field systems due to finite-N effects. The thermalization process involves notably…
The starting point for this work is a static macroscopic model for a high-contrast layered material in single-slip finite crystal plasticity, identified in [Christowiak & Kreisbeck, Calc. Var. PDE (2017)] as a homogenization limit via…
Homology decomposition techniques are a powerful tool used in the analysis of the homotopy theory of (classifying) spaces. The associated Bousfield-Kan spectral sequences involve higher derived limits of the inverse limit functor. We study…
The thermodynamics and the dynamics of particle systems with infinite-range coupling display several unusual and new features with respect to systems with short-range interactions. The Hamiltonian Mean Field (HMF) model represents a…
Transformers have reshaped machine learning by utilizing attention mechanisms to capture complex patterns in large datasets, leading to significant improvements in performance. This success has contributed to the belief that "bigger means…
The dynamics of a degree of freedom associated to an axial vector in contact with a heat bath is decribed by means of a probability distribution function obeying a Fokker-Planck equation. The equation is derived by using mesoscopic…
Transformers are extremely successful machine learning models whose mathematical properties remain poorly understood. Here, we rigorously characterize the behavior of transformers with hardmax self-attention and normalization sublayers as…
In the mean field integrate-and-fire model, the dynamics of a typical neuron within a large network is modeled as a diffusion-jump stochastic process whose jump takes place once the voltage reaches a threshold. In this work, the main goal…
In the mean field integrate-and-fire model, the dynamics of a typical neuron within a large network is modeled as a diffusion-jump stochastic process whose jump takes place once the voltage reaches a threshold. In this work, the main goal…
Although coarse-grained models have been widely used to explain exotic phenomena in complex fluids, such as droplet formation in living cells, these conventional approaches often fail to capture the intricate microscopic degrees of freedom…
With recently developed tools, we prove a homogenisation theorem for a random ODE with short and long-range dependent fractional noise. The effective dynamics are not necessarily diffusions, they are given by stochastic differential…
We consider a diffusion process with coefficients that are periodic outside of an 'interface region' of finite thickness. The question investigated in the articles [1,2] is the limiting long time / large scale behaviour of such a process…
We use dynamic light scattering and computer simulations to study equilibrium dynamics and dynamic heterogeneity in concentrated suspensions of colloidal hard spheres. Our study covers an unprecedented density range and spans seven decades…
Despite the Transformer's dominance across machine learning, its architecture remains largely heuristic and lacks a unified theoretical foundation. We introduce Score-based Variational Flow (SVFlow), a continuous-time dynamical system for…
We study closed dense collections of hard spheres that collide inelastically with constant coefficient of normal restitution. We find inhomogeneous states (IS) where the density profile is spatially non-uniform but constant in time. The…
The homogeneous Boltzmann equation for inelastic Maxwell mixtures is considered to study the dynamics of tracer particles or impurities (solvent) immersed in a uniform granular gas (solute). The analysis is based on exact results derived…
This article is concerned with the fluctuations and the concentration properties of a general class of discrete generation and mean field particle interpretations of nonlinear measure valued processes. We combine an original stochastic…
Highly concentrated patterns have been observed in a spatially heterogeneous, nonlocal, model of BGK type implementing a velocity-jump process. We study both a linear and a nonlinear case and describe the concentration profile. In…
We model the evolution of tokens within a deep stack of Transformer layers as a continuous-time flow on the unit sphere, governed by a mean-field interacting particle system, building on the framework introduced in (Geshkovski et al.,…