Related papers: Kullback-Leibler Potential for Non-Ergodic Replica…
Thermodynamics is based on the notions of energy and entropy. While energy is the elementary quantity governing physical dynamics, entropy is the fundamental concept in information theory. In this work, starting from first principles, we…
The emergence of irreversibility in physical processes, despite the fundamentally reversible nature of quantum mechanics, remains an open question in physics. This thesis explores the intricate relationship between quantum mechanics and…
We explore the dynamics of information systems. We show that the driving force for information dynamics is determined by both the information landscape and information flux which determines the equilibrium time reversible and the…
The integrated information theory is thought to be a key clue towards the theoretical understanding of consciousness. In this study, we propose a simple numerical model comprising a set of coupled double quantum dots, where the…
The present paper proposes an adaptive biasing potential for the computation of free energy landscapes. It is motivated by statistical learning arguments and unifies the tasks of biasing the molecular dynamics to escape free energy wells…
We propose a greedy mixture reduction algorithm which is capable of pruning mixture components as well as merging them based on the Kullback-Leibler divergence (KLD). The algorithm is distinct from the well-known Runnalls' KLD based method…
The Kullback-Leibler divergence offers an information-theoretic basis for measuring the difference between two given distributions. Its quantum analog, however, fails to play a corresponding role for comparing two density matrices, if the…
With the passage of the Right to Be Forgotten (RTBF) regulations and the scaling up of language model training datasets, research on model unlearning in large language models (LLMs) has become more crucial. Before the era of LLMs, machine…
There have been two distinct formalisms of thermodynamics of information: One is the measurement-feedback formalism, which concerns bipartite systems with measurement and feedback processes, and the other is the information reservoir…
We consider a Brownian particle confined by an external potential and subject to stochastic resetting to the origin. Motivated by the repetitive nature of the dynamics, we describe the process as a thermodynamic cycle of thermal expansion…
We study the thermodynamic properties induced by non-reciprocal interactions between stochastic degrees of freedom in time- and space-continuous systems. We show that, under fairly general conditions, non-reciprocal coupling alone implies a…
Drawing on an analogy with the second law of thermodynamics for adiabatically isolated systems, Cover argued that data-processing inequalities may be seen as second laws for "computationally isolated systems," namely, systems evolving…
We study a dynamical system with time dependent Hamiltonian by numerical experiments so as to find a relation between thermodynamics and chaotic nature of the system. Excess information loss, defined newly based on Lyapunov analysis, is…
Landauer argued that the process of erasing the information stored in a memory device incurs an energy cost in the form of a minimum amount of mechanical work. We find, however, that this energy cost can be reduced to zero by paying a cost…
I argue here that both the first and second laws of thermodynamics, generally understood to be quintessentially physical in nature, can be equally well described as being about certain types of information without the need to invoke…
The development of stochastic thermodynamics during the last decades prompted the discovery of novel nonequilibrium relations refining our understanding of the second law in small fluctuating systems and its connection with information…
A common failure mode of density models trained as variational autoencoders is to model the data without relying on their latent variables, rendering these variables useless. Two contributing factors, the underspecification of the model and…
We review recent work on the foundations of thermodynamics in the light of quantum information theory. We adopt a resource-theoretic perspective, wherein thermodynamics is formulated as a theory of what agents can achieve under a particular…
Knowledge distillation (KD), transferring knowledge from a cumbersome teacher model to a lightweight student model, has been investigated to design efficient neural architectures. Generally, the objective function of KD is the…
Formalising the confrontation of opinions (models) to observations (data) is the task of Inferential Statistics. Information Theory provides us with a basic functional, the relative entropy (or Kullback-Leibler divergence), an asymmetrical…