Related papers: Stochastic Thermodynamics for Autoregressive Gener…
The appealing theoretical measure of irreversibility in a stochastic process, as the ratio of the probabilities of a trajectory and its time reversal, cannot be accessed directly in experiment since the probability of a single trajectory is…
Diffusion models recently developed for generative AI tasks can produce high-quality samples while still maintaining diversity among samples to promote mode coverage, providing a promising path for learning stochastic closure models.…
Modern generative machine learning models demonstrate surprising ability to create realistic outputs far beyond their training data, such as photorealistic artwork, accurate protein structures, or conversational text. These successes…
We study stochastic thermodynamics of over-damped Brownian motion in a flowing fluid. Unlike some previous works, we treat the effects of the flow field as a non-conservational driving force acting on the Brownian particle. This allows us…
A positive rate of entropy production at steady state is a distinctive feature of truly non-equilibrium processes. Exact results, while being often limited to simple models, offer a unique opportunity to explore the thermodynamic features…
Estimating the difficulty of input questions as perceived by large language models (LLMs) is essential for accurate performance evaluation and adaptive inference. Existing methods typically rely on repeated response sampling, auxiliary…
The present paper aims to demonstrate the usage of Convolutional Neural Networks as a generative model for stochastic processes, enabling researchers from a wide range of fields (such as quantitative finance and physics) to develop a…
We consider a Brownian particle confined by an external potential and subject to stochastic resetting to the origin. Motivated by the repetitive nature of the dynamics, we describe the process as a thermodynamic cycle of thermal expansion…
We formalise recursive self-training in Large Language Models (LLMs) and Generative AI as a discrete-time dynamical system. We prove that if the proportion of exogenous, externally grounded signal $\alpha_t$ vanishes asymptotically…
Large Language Model (LLM) based code generation is predominantly formulated as a strictly monotonic process, appending tokens linearly to an immutable prefix. This formulation contrasts to the cognitive process of programming, which is…
Autoregressive generative models play a key role in various language tasks, especially for modeling and evaluating long text sequences. While recent methods leverage stochastic representations to better capture sequence dynamics, encoding…
We study the stochastic thermodynamics of an overdamped harmonic chain, which can be viewed equivalently as a 1D Rouse chain or as an approximate model of single file diffusion. We discuss mainly two levels of description of this system:…
Unlike macroscopic engines, the molecular machinery of living cells is strongly affected by fluctuations. Stochastic Thermodynamics uses Markovian jump processes to model the random transitions between the chemical and configurational…
Identifying the full entropy production of active particles is a challenging task. We introduce a microscopic, thermodynamically consistent model, which leads to active Ornstein-Uhlenbeck statistics in the continuum limit. Our minimal model…
The accurate determination of transport coefficients in numerical simulations is becoming increasingly important in a wide range of applications. Here we consider the linear response in systems driven away from thermal equilibrium into a…
Low dimensional representations of words allow accurate NLP models to be trained on limited annotated data. While most representations ignore words' local context, a natural way to induce context-dependent representations is to perform…
The ability to generate and recognize sequential data is fundamental for autonomous systems operating in dynamic environments. Inspired by the key principles of the brain-predictive coding and the Bayesian brain-we propose a novel…
We discuss entropy production in nonequilibrium steady states by focusing on paths obtained by sampling at regular (small) intervals, instead of sampling on each change of the system's state. This allows us to study directly entropy…
The problem of combined state and input estimation of linear structural systems based on measured responses and a priori knowledge of structural model is considered. A novel methodology using Gaussian process latent force models is proposed…
We introduce Generator Matching, a modality-agnostic framework for generative modeling using arbitrary Markov processes. Generators characterize the infinitesimal evolution of a Markov process, which we leverage for generative modeling in a…