Related papers: Erasure entropies and Gibbs measures
This work studies the entropic regularization formulation of the 2-Wasserstein distance on an infinite-dimensional Hilbert space, in particular for the Gaussian setting. We first present the Minimum Mutual Information property, namely the…
Starting from the full--shift on a finite alphabet $A$, mingling some symbols of $A$, we obtain a new full shift on a smaller alphabet $B$. This amalgamation defines a factor map from $(A^{\mathbb N},T_A)$ to $(B^{\mathbb N},T_B)$, where…
Entropy is one of the most fundamental quantities in physics. For systems with few degrees of freedom, the value of entropy provides a powerful insight into its microscopic dynamics, such as the number, degeneracy and relative energies of…
Let $K = \{0,1,...,q-1\}$. We use a special class of translation invariant measures on $K^\mathbb{Z}$ called algebraic measures to study the entropy rate of a hidden Markov processes. Under some irreducibility assumptions of the Markov…
We investigate multiscale Gibbs measures from a variational and probabilistic viewpoint, focusing on the structural asymmetry among conditional entropies that characterizes their construction. We show how this asymmetry emerges both from…
The entropy of an ergodic finite-alphabet process can be computed from a single typical sample path x_1^n using the entropy of the k-block empirical probability and letting k grow with $n$ roughly like log n. We further assume that the…
In this work we consider a problem related to the equilibrium statistical mechanics of spin glasses, namely the study of the Gibbs measure of the random energy model. For solving this problem, new results of independent interest on sums of…
Motivated by the entropy computations relevant to the evaluation of decrease in entropy in bit reset operations, the authors investigate the deficit in an entropic inequality involving two independent random variables, one continuous and…
The entropy of a binary symmetric Hidden Markov Process is calculated as an expansion in the noise parameter epsilon. We map the problem onto a one-dimensional Ising model in a large field of random signs and calculate the expansion…
As no heat effect and mechanical work are observed, we have a simple experimental resolution of the Gibbs paradox: both the thermodynamic entropy of mixing and the Gibbs free energy change are zero during the formation of any ideal…
We expand upon a natural analogy between Bayesian statistics and statistical physics in which sample size corresponds to inverse temperature. This analogy motivates the definition of two novel statistical quantities: a learning capacity and…
Entropy measures have become increasingly popular as an evaluation metric for complexity in the analysis of time series data, especially in physiology and medicine. Entropy measures the rate of information gain, or degree of regularity in a…
In this paper, we extend Gibbs's approach of quasi-equilibrium thermodynamic processes, and calculate the microscopic expression of entropy for general non-equilibrium thermodynamic processes. Also, we analyze the formal structure of…
Neural networks have dramatically increased our capacity to learn from large, high-dimensional datasets across innumerable disciplines. However, their decisions are not easily interpretable, their computational costs are high, and building…
The Gibbs entropy of a macroscopic classical system is a function of a probability distribution over phase space, i.e., of an ensemble. In contrast, the Boltzmann entropy is a function on phase space, and is thus defined for an individual…
We apply a common measure of randomness, the entropy, in the context of iterated functions on a finite set with n elements. For a permutation, it turns out that this entropy is asymptotically (for a growing number of iterations) close to…
Entropy is the measure of uncertainty in any data and is adopted for maximisation of mutual information in many remote sensing operations. The availability of wide entropy variations motivated us for an investigation over the suitability…
In any general cycle of measurement, feedback and erasure, the measurement will reduce the entropy of the system when information about the state is obtained, while erasure, according to Landauer's principle, is accompanied by a…
The transfer entropy is a well-established measure of information flow, which quantifies directed influence between two stochastic time series and has been shown to be useful in a variety fields of science. Here we introduce the transfer…
Statistical inference can be seen as information processing involving input information and output information that updates belief about some unknown parameters. We consider the Bayesian framework for making inferences about dynamical…