Related papers: Stochastic Thermodynamics for Autoregressive Gener…
Stochastic processes generated by non-stationary distributions are difficult to represent with conventional models such as Gaussian processes. This work presents Recurrent Autoregressive Flows as a method toward general stochastic process…
We tackle the question of whether Large Language Models (LLMs), viewed as dynamical systems with state evolving in the embedding space of symbolic tokens, are observable. That is, whether there exist multiple 'mental' state trajectories…
Textless spoken language models (SLMs) are generative models of speech that do not rely on text supervision. Most textless SLMs learn to predict the next semantic token, a discrete representation of linguistic content, and rely on a…
The selection of an equilibrium state by maximising the entropy of a system, subject to certain constraints, is often powerfully motivated as an exercise in logical inference, a procedure where conclusions are reached on the basis of…
The entropy production rate is central to the study of non-equilibrium systems. This parameter is closely connected to violation of time-reversal symmetry, energy consumption, efficiency, and other properties of interest; in short, it…
Generative modeling, which learns joint probability distribution from data and generates samples according to it, is an important task in machine learning and artificial intelligence. Inspired by probabilistic interpretation of quantum…
Stochastic thermodynamics is the field of study relating fluctuations in stochastic systems to thermodynamic quantities. The total entropy production (EP), is central to the thermodynamic classification of systems. Non-equilibrium systems…
Monitoring the spatiotemporal dynamics of forest CO$_2$ uptake (Gross Primary Production, GPP), remains a central challenge in terrestrial ecosystem research. While Eddy Covariance (EC) towers provide high-frequency estimates, their limited…
A set of core features is set forth as the essence of a thermodynamic description, which derive from large-deviation properties in systems with hierarchies of timescales, but which are \emph{not} dependent upon conservation laws or…
Developing a thermodynamic theory of computation is a challenging task at the interface of non-equilibrium thermodynamics and computer science. In particular, this task requires dealing with difficulties such as stochastic halting times,…
A syntactic language model (SLM) incrementally generates a sentence with its syntactic tree in a left-to-right manner. We present Generative Pretrained Structured Transformers (GPST), an unsupervised SLM at scale capable of being…
Interpreting partial information collected from systems subject to noise is a key problem across scientific disciplines. Theoretical frameworks often focus on the dynamics of variables that result from coarse-graining the internal states of…
Generative models have demonstrated remarkable success in domains such as text, image, and video synthesis. In this work, we explore the application of generative models to fluid dynamics, specifically for turbulence simulation, where…
We investigate the entropic consequences of the relaxation of an open two-level quantum system towards a thermalised statistical state, using a framework of quantum state diffusion with a minimal set of raising and lowering Lindblad…
Stochastic simulation models effectively capture complex system dynamics but are often too slow for real-time decision-making. Traditional metamodeling techniques learn relationships between simulator inputs and a single output summary…
In this work we introduce and characterize a broad class of quantum operations with a unique fixed point, termed quantum ergodic channels. We derive Lindblad-type master equations for these channels in arbitrary finite dimensions and…
Discrete automated processes in industrial and cyber-physical systems often exhibit a repetitive structure in which successive repetitions follow a common trajectory while differing in duration, amplitude, and fine-scale dynamics. Such…
We investigate the total stochastic entropy production of a two-level bosonic open quantum system under protocols of time dependent coupling to a harmonic environment. These processes are intended to represent the measurement of a system…
We propose a general framework for modelling network data that is designed to describe aspects of non-exchangeable networks. Conditional on latent (unobserved) variables, the edges of the network are generated by their finite growth history…
Entropy production and the detailed fluctuation theorem are of fundamental importance for thermodynamic processes. In this paper, we study the multiple entropy production for multitime quantum processes in a unified framework. For closed…