Related papers: Entropy Production in Machine Learning Under Fokke…

We interpret a class of nonlinear Fokker-Planck equations with reaction as gradient flows over the space of Radon measures equipped with the recently introduced Hellinger-Kantorovich distance. The driving entropy of the gradient flow is not…

A theory of neural networks (NNs) built upon collective variables would provide scientists with the tools to better understand the learning process at every stage. In this work, we introduce two such variables, the entropy and the trace of…

Using the stochastic thermodynamics, we determine the entropy production and the dynamic heat capacity of systems subject to a sinusoidally time dependent temperature, in which case the systems are permanently out of thermodynamic…

Decoding strategies play a central role in shaping the reasoning ability of large language models (LLMs). Traditional methods such as greedy decoding and beam search often suffer from error propagation, while sampling-based approaches…

Most predictive models assume that training and test data are generated from a stationary process. However, this assumption does not hold true in practice. In this paper, we consider the scenario of a gradual concept drift due to the…

While data selection methods have been studied extensively in active learning, data pruning, and data augmentation settings, there is little evidence for the efficacy of these methods in industry scale settings, particularly in low-resource…

Statistical learning under distributional drift remains poorly characterized, especially in closed-loop settings where learning alters the data-generating law. We introduce an intrinsic drift budget $C_T$ that quantifies cumulative…

In this paper we show that the expected generalisation performance of a learning machine is determined by the distribution of risks or equivalently its logarithm -- a quantity we term the risk entropy -- and the fluctuations in a quantity…

We study how to allocate resources for training and deployment of machine learning (ML) models under concept drift and limited budgets. We consider a setting in which a model provider distributes trained models to multiple clients whose…

We present a simple comparative framework for testing and developing uncertainty modeling in uncertain marching cubes implementations. The selection of a model to represent the probability distribution of uncertain values directly…

We study the current rectification of particles moving in a pulsating channel under the in uence of an applied force. We have shown the existence of diferent rectification scenarios in which entropic and energetic effects compete. The…

Model-free Reinforcement Learning (RL) works well when experience can be collected cheaply and model-based RL is effective when system dynamics can be modeled accurately. However, both assumptions can be violated in real world problems such…

A restricted Boltzmann machine is a generative probabilistic graphic network. A probability of finding the network in a certain configuration is given by the Boltzmann distribution. Given training data, its learning is done by optimizing…

This study examines a new formulation of non-equilibrium thermodynamics, which gives a conditional derivation of the ``maximum entropy production'' (MEP) principle for flow and/or chemical reaction systems at steady state. The analysis uses…

We present an iterative inverse reinforcement learning algorithm to infer optimal cost functions in continuous spaces. Based on a popular maximum entropy criteria, our approach iteratively finds a weight improvement step and proposes a…

Coupling physics with machine learning models has shown great potential for solving fluid dynamics problems governed by partial differential equations. However, conventional methods, such as physics-informed neural networks, often suffer…

This study presents a conditional flow matching framework for solving physics-constrained Bayesian inverse problems. In this setting, samples from the joint distribution of inferred variables and measurements are assumed available, while…

Computing the stochastic entropy production associated with the evolution of a stochastic dynamical system is a well-established problem. In a small number of cases such as the Ornstein-Uhlenbeck process, of which we give a complete…

We study the recurrence to mistake dynamical balls, that is, dynamical balls that admit some errors and whose proportion of errors decrease tends to zero with the length of the dynamical ball. We prove, under mild assumptions, that the…

Frontier reasoning models are produced by posttraining base language models with reinforcement learning. Recent work has challenged this by showing that sampling from a sharpened version of the base model's distribution, a so-called power…