Related papers: Stochastic Thermodynamics for Autoregressive Gener…
For a network of discrete states with a periodically driven Markovian dynamics, we develop an inference scheme for an external observer who has access to some transitions. Based on waiting-time distributions between these transitions, the…
The consistency across scales of a recently developed mathematical thermodynamic structure, between a continuous stochastic nonlinear dynamical system (diffusion process with Langevin or Fokker-Planck equations) and its emergent discrete,…
Small systems in a thermodynamic medium --- like colloids in a suspension or the molecular machinery in living cells --- are strongly affected by the thermal fluctuations of their environment. Physicists model such systems by means of…
Living systems are open nonequilibrium systems that continuously exchange energy, matter, and information with their environments, leading to stochastic dynamics with memory and active fluctuations. In this study, we develop a non-Markovian…
Recently, Transformer based pretrained language models (PLMs), such as GPT2 and T5, have been leveraged to build generative task-oriented dialog (TOD) systems. A drawback of existing PLM-based models is their non-Markov architectures across…
Conventional predictive modeling of parametric relationships in manufacturing processes is limited by the subjectivity of human expertise and intuition on the one hand and by the cost and time of experimental data generation on the other…
Non-autoregressive generative transformers recently demonstrated impressive image generation performance, and orders of magnitude faster sampling than their autoregressive counterparts. However, optimal parallel sampling from the true joint…
Understanding the entropy production of systems strongly coupled to thermal baths is a core problem of both quantum thermodynamics and mesoscopic physics. While there exist many techniques to accurately study entropy production in such…
We examine stochastic processes that are used to model nonequilibrium processes (e.g, pulling RNA or dragging colloids) and so deliberately violate detailed balance. We argue that by combining an information-theoretic measure of…
Understanding the temporal dynamics of Earth's surface is a mission of multi-temporal remote sensing image analysis, significantly promoted by deep vision models with its fuel -- labeled multi-temporal images. However, collecting,…
We establish a fundamental connection between score-based diffusion models and non-equilibrium thermodynamics by deriving performance limits based on entropy rates. Our main theoretical contribution is a lower bound on the negative…
Autoregressive language models (LMs) generate one token at a time, yet human reasoning operates over higher-level abstractions - sentences, propositions, and concepts. This contrast raises a central question- Can LMs likewise learn to…
The dominant language modeling paradigm handles text as a sequence of discrete tokens. While that approach can capture the latent structure of the text, it is inherently constrained to sequential dynamics for text generation. We propose a…
This PhD thesis deals with the Markov picture of developed turbulence from the theoretical point of view. The thesis consists of two parts. The first part introduces stochastic thermodynamics, the second part aims at transferring the…
The increasing prevalence of Large Language Models (LMs) in critical applications highlights the need for controlled language generation strategies that are not only computationally efficient but that also enjoy performance guarantees. To…
We study the multiple definitions of the entropy production for discrete-time Markov processes in single systems and composite systems. These definitions have been studied in single systems, but less so in composite systems. With a clear…
We propose a latent score-based generative AI framework for learning stochastic, non-local closure models and constitutive laws in nonlinear dynamical systems of computational mechanics. This work addresses a key challenge of modeling…
Recent studies suggest utilizing generative models instead of traditional auto-regressive algorithms for time series forecasting (TSF) tasks. These non-auto-regressive approaches involving different generative methods, including GAN,…
Stochastic thermodynamics investigates energetic and entropic bounds in small systems. Foundational results, e.g., the first and second laws, predominantly rely on the Markov (memoryless) assumption. Although physicists recognise that the…
This paper presents an {\it ab initio} derivation of the expression given by irreversible thermodynamics for the rate of entropy production for different classes of diffusive processes. The first class are Lorentz gases, where…