Related papers: On the correspondence between thermodynamics and i…

We introduce a new, rigorously-formulated Bayesian meta-learning algorithm that learns a probability distribution of model parameter prior for few-shot learning. The proposed algorithm employs a gradient-based variational inference to infer…

This monograph deals with adaptive supervised classification, using tools borrowed from statistical mechanics and information theory, stemming from the PACBayesian approach pioneered by David McAllester and applied to a conception of…

In this article, we review a general theoretical framework of thermodynamics of information on the basis of Bayesian networks. This framework can describe a broad class of nonequilibrium dynamics of multiple interacting systems with complex…

The large-deviation method allows to characterize an ergodic counting process in terms of a thermodynamic frame where a free energy function determines the asymptotic non-stationary statistical properties of its fluctuations. Here, we study…

We introduce a simple and intuitive framework that provides quantitative explanations of statistical models through the probabilistic assessment of input feature importance. The core idea comes from utilizing the Dirichlet distribution to…

Bayesian inference is often utilized for uncertainty quantification tasks. A recent analysis by Xu and Raginsky 2022 rigorously decomposed the predictive uncertainty in Bayesian inference into two uncertainties, called aleatoric and…

There are three principle paradigms of statistical inference: (i) Bayesian, (ii) information-based and (iii) frequentist inference. We describe an objective prior (the weighting or $w$-prior) which unifies objective Bayes and…

In this paper, a mathematical theory of learning is proposed that has many parallels with information theory. We consider Vapnik's General Setting of Learning in which the learning process is defined to be the act of selecting a hypothesis…

Bayesian hierarchical models are frequently used in practical data analysis contexts. One interpretation of these models is that they provide an indirect way of assigning a prior for unknown parameters, through the introduction of…

A quantum statistical expression for the entropy of a nonequilibrium system is defined so as to be consistent with Gibbs' relation, and is shown to corresponds to dynamical variable by introducing analogous to the Heisenberg picture in…

We prove a lower bound on the relative entropy between two finite-dimensional states in terms of their entropy difference and the dimension of the underlying space. The inequality is tight in the sense that equality can be attained for any…

I discuss in parallel two universal phenomena: the independence of statistics of the heat capacity and entropy of ideal gases of the same, constant, density of states, on one hand, and the independence of statistics of the heat and entropy…

Score-based methods, such as diffusion models and Bayesian inverse problems, are often interpreted as learning the data distribution in the low-noise limit ($\sigma \to 0$). In this work, we propose an alternative perspective: their success…

Universality of classical thermodynamics rests on the central limit theorem, due to which, measurements of thermal fluctuations are unable to reveal detailed information regarding the microscopic structure of a macroscopic body. When small…

Superstatistics is a framework in nonequilibrium statistical mechanics that successfully describes a wide variety of complex systems, including hydrodynamic turbulence, weakly-collisional plasmas, cosmic rays, power grid fluctuations, among…

To understand under which conditions thermodynamics emerges from the microscopic dynamics is the ultimate goal of statistical mechanics. Despite the fact that the theory is more than 100 years old, we are still discussing its foundations…

Thermodynamics as limiting behaviors of statistics is generalized to arbitrary system with probability {\it a priori} where thermodynamic infinite-size limit is replaced by multiple-measurement limit. A duality symmetry between Massieu's…

As no heat effect and mechanical work are observed, we have a simple experimental resolution of the Gibbs paradox: both the thermodynamic entropy of mixing and the Gibbs free energy change are zero during the formation of any ideal…

Despite exceptional predictive performance of Deep sequence models (DSMs), the main concern of their deployment centers around the lack of uncertainty awareness. In contrast, probabilistic models quantify the uncertainty associated with…

A pedagogical approach for deriving the statistical mechanical partition function, in a manner that emphasizes the key role of entropy in connecting the microscopic states to thermodynamics, is introduced. The connections between the…