Related papers: Large Deviations and Metastability Analysis for He…
We develop a unified theory to analyze the microcanonical ensembles with several constraints given by unbounded observables. Several interesting phenomena that do not occur in the single constraint case can happen under the multiple…
In recent years, various notions of capacity and complexity have been proposed for characterizing the generalization properties of stochastic gradient descent (SGD) in deep learning. Some of the popular notions that correlate well with the…
The large deviation principle is proved for a class of $L^2$-valued processes that arise from the coarse-graining of a random field. Coarse-grained processes of this kind form the basis of the analysis of local mean-field models in…
We prove pathwise large deviation principles of slow variables in slow-fast systems in the limit of time-scale separation tending to infinity. In the limit regime we consider, the convergence of the slow variable to its deterministic limit…
In ergodic physical systems, time-averaged quantities converge (for large times) to their ensemble-averaged values. Large deviation theory describes rare events where these time averages differ significantly from the corresponding ensemble…
Stochastic approximation (SA) is a fundamental iterative framework with broad applications in reinforcement learning and optimization. Classical analyses typically rely on martingale difference or Markov noise with bounded second moments,…
Using the weak convergence approach, we prove the large deviation principle (LDP) for solutions to quasilinear stochastic evolution equations with small Gaussian noise in the critical variational setting, a recently developed general…
We often rely on probabilistic measures -- e.g. event probability or expected time -- to characterize systems' safety. However, determining these quantities for extremely low-probability events is generally challenging, as standard safety…
In existing distributed stochastic optimization studies, it is usually assumed that the gradient noise has a bounded variance. However, recent research shows that the heavy-tailed noise, which allows an unbounded variance, is closer to…
Motivated by the study of the time evolution of random dynamical systems arising in a vast variety of domains --- ranging from physics to ecology ---, we establish conditions for the occurrence of a non-trivial asymptotic behaviour for…
Although stochastic optimization is central to modern machine learning, the precise mechanisms underlying its success, and in particular, the precise role of the stochasticity, still remain unclear. Modelling stochastic optimization…
We consider generalized Bayesian inference on stochastic processes and dynamical systems with potentially long-range dependency. Given a sequence of observations, a class of parametrized model processes with a prior distribution, and a loss…
We present a review of recent work on the statistical mechanics of non equilibrium processes based on the analysis of large deviations properties of microscopic systems. Stochastic lattice gases are non trivial models of such phenomena and…
The theory of large deviations constitutes a mathematical cornerstone in the foundations of Boltzmann-Gibbs statistical mechanics, based on the additive entropy $S_{BG}=- k_B\sum_{i=1}^W p_i \ln p_i$. Its optimization under appropriate…
Large-deviations theory deals with tails of probability distributions and the rare events of random processes, for example spreading packets of particles. Mathematically, it concerns the exponential fall-of of the density of thin-tailed…
This paper introduces a new classification scheme - head/tail breaks - in order to find groupings or hierarchy for data with a heavy-tailed distribution. The heavy-tailed distributions are heavily right skewed, with a minority of large…
We provide an overview of high dimensional dynamical systems driven by random matrices, focusing on applications to simple models of learning and generalization in machine learning theory. Using both cavity method arguments and path…
How do large deviation events in a stationary process cluster? The answer depends not only on the type of large deviations, but also on the length of memory in the process. Somewhat unexpectedly, it may also depend on the tails of the…
Deploying deep models in real-world scenarios entails a number of challenges, including computational efficiency and real-world (e.g., long-tailed) data distributions. We address the combined challenge of learning long-tailed distributions…
Convolutions of long-tailed and subexponential distributions play a major role in the analysis of many stochastic systems. We study these convolutions, proving some important new results through a simple and coherent approach, and showing…