Related papers: Stochastic Thermodynamics for Autoregressive Gener…

Generative machine learning models offer a powerful framework for therapeutic design by efficiently exploring large spaces of biological sequences enriched for desirable properties. Unlike supervised learning methods, which require both…

We consider a one-dimensional persisent random walk viewed as a deterministic process with a form of time reversal symmetry. Particle reservoirs placed at both ends of the system induce a density current which drives the system out of…

Non-Markovian dynamics are ubiquitous across physics, biology, and engineering. Yet our understanding of non-Markovian processes significantly lags that of simpler Markovian processes, due largely to a lack of tractable models. In this…

The problem of time-series forecasting in non-stationary and complex environments is a challenging task in machine learning, especially with heterogeneous numerical and textual data present. Traditional statistical models like…

We study the relationship between (non-)Markovian evolutions, established correlations, and the entropy production rate. We consider a system qubit in contact with a thermal bath and in addition the system is strongly coupled to an…

Continuous feedback control of Langevin processes may be non-Markovian due to a time lag between the measurement and the control action. We show that this requires to modify the basic relation between dissipation and time-reversal and to…

Continuous-time Markov chains are used to model stochastic systems where transitions can occur at irregular times, e.g., birth-death processes, chemical reaction networks, population dynamics, and gene regulatory networks. We develop a…

This paper investigates the theoretical behavior of generative models under finite training populations. Within the stochastic interpolation generative framework, we derive closed-form expressions for the optimal velocity field and score…

In contexts where data samples represent a physically stable state, it is often assumed that the data points represent the local minima of an energy landscape. In control theory, it is well-known that energy can serve as an effective…

With latent variables, stochastic recurrent models have achieved state-of-the-art performance in modeling sound-wave sequence. However, opposite results are also observed in other domains, where standard recurrent networks often outperform…

Recent advances in analysis of subband amplitude envelopes of natural sounds have resulted in convincing synthesis, showing subband amplitudes to be a crucial component of perception. Probabilistic latent variable analysis is particularly…

In all but special circumstances, measurements of time-dependent processes reflect internal structures and correlations only indirectly. Building predictive models of such hidden information sources requires discovering, in some way, the…

The generic behavior of quantum systems has long been of theoretical and practical interest. Any quantum process is represented by a sequence of quantum channels. We consider general ergodic sequences of stochastic channels with arbitrary…

This paper explores the application of Stochastic Differential Equations (SDE) to interpret the text generation process of Large Language Models (LLMs) such as GPT-4. Text generation in LLMs is modeled as a stochastic process where each…

The process of translation is ambiguous, in that there are typically many valid trans- lations for a given sentence. This gives rise to significant variation in parallel cor- pora, however, most current models of machine translation do not…

Data augmentation is gaining importance across various aspects of time series analysis, from forecasting to classification and anomaly detection tasks. We introduce the Latent Generative Transformer Augmentation (L-GTA) model, a generative…

This article is concerned with learning and stochastic control in physical systems which contain unknown input signals. These unknown signals are modeled as Gaussian processes (GP) with certain parametrized covariance structures. The…

With nontrivial entropy production, first passage process is one of the most common nonequilibrium process in stochastic thermodynamics. Using one dimensional birth and death precess as a model framework, approximated expressions of mean…

Variational Autoencoders are powerful models for unsupervised learning. However deep models with several layers of dependent stochastic variables are difficult to train which limits the improvements obtained using these highly expressive…

Large language models (LLMs) demonstrate the capacity to reconstruct and trace learned content from their training data under specific elicitation conditions, yet this capability does not manifest in standard generation contexts. This…