Related papers: Manifold Trajectories in Next-Token Prediction: Fr…

Inference-time reward alignment asks how to turn a pre-trained diffusion model with base law $p$ into a sampler that favors a reward $r$ while remaining close to $p$. Since there is no canonical distributional distance for this closeness…

Speculative decoding has emerged as an effective approach for accelerating autoregressive inference by parallelizing token generation through a draft-then-verify paradigm. However, existing methods rely on static drafting lengths and rigid…

We propose a novel methodology for general multi-class classification in arbitrary feature spaces, which results in a potentially well-calibrated classifier. Calibrated classifiers are important in many applications because, in addition to…

We combine the swap Monte Carlo algorithm to long multi-CPU molecular dynamics simulations to analyse the equilibrium relaxation dynamics of model supercooled liquids over a time window covering ten orders of magnitude for temperatures down…

While the approximation properties of single-layer Transformer architectures have been studied in recent works, a rigorous theoretical understanding of the multi-layer setting remains limited. In this work, we establish that multi-layer…

Metric learning aims at learning a distance which is consistent with the semantic meaning of the samples. The problem is generally solved by learning an embedding for each sample such that the embeddings of samples of the same category are…

Diffusion models achieve remarkable generation quality, yet face a fundamental challenge known as memorization, where generated samples can replicate training samples exactly. We develop a theoretical framework to explain this phenomenon by…

Experimental evidence indicates that simple models outperform complex deep networks on many unsupervised similarity tasks. We provide a simple yet rigorous explanation for this behaviour by introducing the concept of an optimal…

Generative modeling is typically framed as learning mapping rules, but from an observer's perspective without access to these rules, the task becomes disentangling the geometric support from the probability distribution. We propose that…

We propose a method for learning and sampling from probability distributions supported on the simplex. Our approach maps the open simplex to Euclidean space via smooth bijections, leveraging the Aitchison geometry to define the mappings,…

We study, both numerically and analytically, the development of equilibrium after preheating. We show that the process is characterised by the appearance of Kolmogorov spectra and the evolution towards thermal equilibrium follows…

Demixing of binary fluids subjected to slow temperature ramps shows repeated waves of nucleation which arise as a consequence of the competition between generation of supersaturation by the temperature ramp and relaxation of supersaturation…

While diffusion models generate high-quality images via probability flow, the theoretical understanding of this process remains incomplete. A key question is when probability flow converges to training samples or more general points on the…

Improved mean-field technics are a central theme of statistical physics methods applied to inference and learning. We revisit here some of these methods using high-temperature expansions for disordered systems initiated by Plefka, Georges…

Light diffusion is usually associated with thick, opaque media. Indeed, multiple scattering is necessary for the onset of the diffusive regime and such condition is generally not met in almost transparent media. Nonetheless, at long enough…

The threat of global warming and the demand for reliable climate predictions pose a formidable challenge being the climate system multiscale, high-dimensional and nonlinear. Spatiotemporal recurrences of the system hint to the presence of a…

Markov chain Monte Carlo (MCMC) methods provide powerful framework for sampling unknown probability measures across a wide range of scientific applications. In some settings, the target distribution is supported on a lower-dimensional…

This article is devoted to the long-time dynamics of point-vortex type systems near thermal equilibrium and to the possible emergence of collisional relaxation. More precisely, we consider a tagged particle coupled to a large number of…

The long-term dynamics of many dynamical systems evolve on an attracting, invariant "slow manifold" that can be parameterized by a few observable variables. Yet a simulation using the full model of the problem requires initial values for…

Next-token prediction (NTP) over large text corpora has become the go-to paradigm to train large language models. Yet, it remains unclear how NTP influences the mapping of linguistic patterns to geometric properties of the resulting model…