Related papers: Universal Approximation of Mean-Field Models via T…

We introduce a family of glassy models having a parameter, playing the role of an interaction range, that may be varied continuously to go from a system of particles in d dimensions to a mean-field version of it. The mean-field limit is…

For algorithms based on interacting particle systems that admit a mean-field description, convergence analysis is often more accessible at the mean-field level. In order to transfer convergence results obtained at the mean-field level to…

We develop a mathematical framework that interprets Transformer attention as an interacting particle system and studies its continuum (mean-field) limits. By idealizing attention on the sphere, we connect Transformer dynamics to Wasserstein…

The mean-field approximations of many-boson dynamics are known to be effective in many physical relevant situations. The mathematical justifications of such approximations rely generally on specific considerations which depend too much on…

We use probabilistic methods to study properties of mean-field models, arising as large-scale limits of certain particle systems with mean-field interaction. The underlying particle system is such that $n$ particles move forward on the real…

Interacting particle systems are in frequent use to model collective behaviour in various situations and applications. For many systems, the interaction between the agents is restricted to an underlying network structure and often, the…

Many real-world phenomena can be modelled as dynamical processes on networks, a prominent example being the spread of infectious diseases such as COVID-19. Mean-field approximations are a widely used tool to analyse such dynamical processes…

We analyze an algorithm to numerically solve the mean-field optimal control problems by approximating the optimal feedback controls using neural networks with problem specific architectures. We approximate the model by an $N$-particle…

Transformer-based models have demonstrated exceptional performance across diverse domains, becoming the state-of-the-art solution for addressing sequential machine learning problems. Even though we have a general understanding of the…

In this paper, we study the evolution of tokens through the depth of encoder-only transformer models at inference time by modeling them as a system of particles interacting in a mean-field way and studying the corresponding dynamics. More…

We simulate the collective dynamics in spin lattices with long range interactions and collective decay in one, two and three dimensions. Starting from a dynamical mean-field approach derived by local factorization of the density operator we…

Large language models are capable of in-context learning, the ability to perform new tasks at test time using a handful of input-output examples, without parameter updates. We develop a universal approximation theory to elucidate how…

Transformers are widely used in natural language processing due to their ability to model longer-term dependencies in text. Although these models achieve state-of-the-art performance for many language related tasks, their applicability…

Mean-field stochastic differential equations, also called McKean--Vlasov equations, are the limiting equations of interacting particle systems with fully symmetric interaction potential. Such systems play an important role in a variety of…

Despite the widespread adoption of Transformer models for NLP tasks, the expressive power of these models is not well-understood. In this paper, we establish that Transformer models are universal approximators of continuous permutation…

We consider a large number $N$ of quantum particles coupled via a mean field interaction to another quantum system (reservoir). Our main result is an expansion for the averages of observables, both of the particles and of the reservoir, in…

The mean field algorithm is a widely used approximate inference algorithm for graphical models whose exact inference is intractable. In each iteration of mean field, the approximate marginals for each variable are updated by getting…

This article proposes a unified framework to study non-exchangeable mean-field particle systems with some general interaction mechanisms. The starting point is a fixed-point formulation of particle systems originally due to Tanaka that…

The paper is concerned with the approximation of the deterministic the mean field type control system by a mean field Markov chain. It turns out that the dynamics of the distribution in the approximating system is described by a system of…

Mean-field analysis is an important tool for understanding dynamics on complex networks. However, surprisingly little attention has been paid to the question of whether mean-field predictions are accurate, and this is particularly true for…