Related papers: On learning higher-order cumulants in diffusion mo…

Point processes model the distribution of random point sets in mathematical spaces, such as spatial and temporal domains, with applications in fields like seismology, neuroscience, and economics. Existing statistical and machine learning…

We consider general multi-species models of reaction diffusion processes and obtain a set of constraints on the rates which give rise to closed systems of equations for correlation functions. Our results are valid in any dimension and on…

The gaussian closure approximation previously used to study the growth kinetics of the non-conserved O(n) model is shown to be the zeroth-order approximation in a well-defined sequence of approximations composing a more elaborate theory.…

In a convergence of machine learning and biology, we reveal that diffusion models are evolutionary algorithms. By considering evolution as a denoising process and reversed evolution as diffusion, we mathematically demonstrate that diffusion…

Diffusion processes are a class of stochastic differential equations (SDEs) providing a rich family of expressive models that arise naturally in dynamic modelling tasks. Probabilistic inference and learning under generative models with…

Linear causal disentanglement is a recent method in causal representation learning to describe a collection of observed variables via latent variables with causal dependencies between them. It can be viewed as a generalization of both…

Understanding how adult humans learn non-native speech categories such as tone information has shed novel insights into the mechanisms underlying experience-dependent brain plasticity. Scientists have traditionally examined these questions…

Noise is a result of stochastic processes that originate from quantum or classical sources. Higher-order cumulants of the probability distribution underlying the stochastic events are believed to contain details that characterize the…

Deep Gaussian Processes learn probabilistic data representations for supervised learning by cascading multiple Gaussian Processes. While this model family promises flexible predictive distributions, exact inference is not tractable.…

Uncovering the mechanisms behind long-term memory is one of the most fascinating open problems in neuroscience and artificial intelligence. Artificial associative memory networks have been used to formalize important aspects of biological…

Diffusion models power leading generative AI, but when and how they memorize training data, especially on low-dimensional manifolds, remains unclear. We find memorization emerges gradually, not abruptly: as data become scarce, diffusion…

We present a class of diffusion-based algorithms to draw samples from high-dimensional probability distributions given their unnormalized densities. Ideally, our methods can transport samples from a Gaussian distribution to a specified…

Nonequilibrium behaviors of positional order are discussed based on diffusion processes in particle systems. With the cumulant expansion method up to the second order, we obtain a relation between the positional order parameter $\Psi$ and…

Diffusive transport in many complex systems features a crossover between anomalous diffusion at short times and normal diffusion at long times. This behavior can be mathematically modeled by cutting off (tempering) beyond a mesoscopic…

Discrete diffusion models have recently become competitive with autoregressive models for language modeling, even outperforming them on reasoning tasks requiring planning and global coherence, but they require more computation at inference…

With the ever-increasing complexity of large-scale pre-trained models coupled with a shortage of labeled data for downstream training, transfer learning has become the primary approach in many fields, including natural language processing,…

We introduce a cumulant-expansion framework for quantifying how large language models (LLMs) internalize higher-order statistical structure during next-token prediction. By treating the softmax entropy of each layer's logit distribution as…

Diffusion models have recently emerged as a powerful framework for generative modeling. They consist of a forward process that perturbs input data with Gaussian white noise and a reverse process that learns a score function to generate…

Diffusion models have emerged as effective distribution estimators in vision, language, and reinforcement learning, but their use as priors in downstream tasks poses an intractable posterior inference problem. This paper studies amortized…

Higher order symmetric cumulants of global collective observables in heavy ion collisions are studied. The symmetric cumulants can be straightforwardly constructed for scalar observables: the average transverse momentum, the multiplicity,…