Related papers: Stochastic Scaling Limits and Synchronization by N…
Transformers have revolutionized deep learning across various domains but understanding the precise token dynamics remains a theoretical challenge. Existing theories of deep Transformers with layer normalization typically predict that…
We consider a system of stochastic interacting particles in $\mathbb{R}^d$ and we describe large deviations asymptotics in a joint mean-field and small-noise limit. Precisely, a large deviations principle (LDP) is established for the…
In this work we determine a process-level Large Deviation Principle (LDP) for a model of interacting particles indexed by a lattice $\mathbb{Z}^d$. The connections are random, sparse and unscaled, so that the system converges in the large…
We study a random model of deep multi-head self-attention in which the weights are resampled independently across layers and heads, as at initialization of training. Viewing depth as a time variable, the residual stream defines a…
We study the large deviation behavior of a system of diffusing particles with a mean field interaction, described through a collection of stochastic differential equations, in which each particle is driven by a vanishing independent…
We demonstrate that waves in distinct layers of a neuronal network can become phase-locked by common spatiotemporal noise. This phenomenon is studied for stationary bumps, traveling waves, and breathers. A weak noise expansion is used to…
A system of interacting particles described by stochastic differential equations is considered. As oppopsed to the usual model, where the noise perturbations acting on different particles are independent, here the particles are subject to…
For a class of coupled limit cycle oscillators, we give a condition on a linear coupling operator that is necessary and sufficient for exponential stability of the synchronous solution. We show that with certain modifications our method of…
A system of interacting multiclass finite-state jump processes is analyzed. The model under consideration consists of a block-structured network with dynamically changing multi-colors nodes. The interaction is local and described through…
We introduce token-consistent stochastic layers in vision transformers, without causing any severe drop in performance. The added stochasticity improves network calibration, robustness and strengthens privacy. We use linear layers with…
Spatially extended dynamical systems, namely coupled map lattices, driven by additive spatio-temporal noise are shown to exhibit stochastic synchronization. In analogy with low-dymensional systems, synchronization can be achieved only if…
There is a fundamental limit on the capacity of fibre optical communication system (Shannon Limit). This limit can be potentially overcome via using Nonlinear Frequency Division Multiplexing. Dealing with noises in these systems is one of…
Interacting particle systems provide a fundamental framework for modeling collective behavior in biological, social, and physical systems. In many applications, stochastic perturbations are essential for capturing environmental variability…
We analyze the strong noise limit of one-dimensional stochastic differential equations (SDEs). Our initial motivation comes from continuous measurements of open quantum systems. In this context, Bauer, Bernard and Tilloy pointed out an…
Consider a collection of particles whose state evolution is described through a system of interacting diffusions in which each particle is driven by an independent individual source of noise and also by a small amount of noise that is…
We consider stochastic inviscid dyadic models with energy-preserving noise. It is shown that the models admit weak solutions which are unique in law. Under a certain scaling limit of the noise, the stochastic models converge weakly to a…
We study the asymptotic behavior of a class of stochastic dynamics on interlacing particle configurations (also known as Gelfand-Tsetlin patterns). Examples of such dynamics include, in particular, a multi-layer extension of TASEP and…
Overdamped stochastic systems maintained far from equilibrium can display sustained oscillations with fluctuations that decrease with the system size. The correlation time of such noisy limit cycles expressed in units of the cycle period is…
Realistic networks display heterogeneous transmission delays. We analyze here the limits of large stochastic multi-populations networks with stochastic coupling and random interconnection delays. We show that depending on the nature of the…
In this paper, we study the evolution of tokens through the depth of encoder-only transformer models at inference time by modeling them as a system of particles interacting in a mean-field way and studying the corresponding dynamics. More…