Related papers: Gelfand-Yaglom-Perez Theorem for Generalized Relat…
This article presents in a self-contained way A. Uhlmann's celebrated Theorem of monotonicity of the relative entropy under completely positive and trace preserving maps. The Theorem is presented in its more general form and meaningful…
Ensemble of initial conditions for nonlinear maps can be described in terms of entropy. This ensemble entropy shows an asymptotic linear growth with rate K. The rate K matches the logarithm of the corresponding asymptotic sensitivity to…
The paper suggests the concepts of an upper entropy and a lower entropy. We propose a new axiomatic definition, namely, upper entropy axioms, inspired by axioms of metric spaces, and also formulate lower entropy axioms. We also develop weak…
This paper provides a unified perspective for the Kullback-Leibler (KL)-divergence and the integral probability metrics (IPMs) from the perspective of maximum likelihood density-ratio estimation (DRE). Both the KL-divergence and the IPMs…
A definition for the entanglement entropy in both Abelian and non-Abelian gauge theories has been given in the literature, based on an extended Hilbert space construction. The result can be expressed as a sum of two terms, a classical term…
The physical foundations of a variety of emerging technologies --- ranging from the applications of quantum entanglement in quantum information to the applications of nonequilibrium bulk and interface phenomena in microfluidics, biology,…
We provide a unifying axiomatics for Renyi's entropy and non-extensive entropy of Tsallis. It is shown that the resulting entropy coincides with Csiszar's measure of directed divergence known from communication theory.
We consider the central limit theorem for stable laws in the case of the standardized sum of independent and identically distributed random variables with regular probability density function. By showing decay of different entropy…
The notion of entropy appears in many fields and this paper is a survey about entropies in several branches of Mathematics. We are mainly concerned with the topological and the algebraic entropy in the context of continuous endomorphisms of…
The Kullback-Leibler (KL) divergence is a fundamental equation of information theory that quantifies the proximity of two probability distributions. Although difficult to understand by examining the equation, an intuition and understanding…
For infinite measure-theoretic entropy systems, we introduce the notion of measure-theoretic metric mean dimension of invariant measures for different types of measure-theoretic $\epsilon$-entropies, and show that measure-theoretic metric…
General characterization of physical measurements is discussed within the framework of a classical information theory. Uncertainty relation for simultaneous measurements of two physical observables is defined in this framework for…
We examine the {combinatorial} or {probabilistic} definition ("Boltzmann's principle") of the entropy or cross-entropy function $H \propto \ln \mathbb{W}$ or $D \propto - \ln \mathbb{P}$, where $\mathbb{W}$ is the statistical weight and…
We investigate fundamental connections between thermodynamics and quantum information theory. First, we show that the operational framework of thermal operations is nonequivalent to the framework of Gibbs-preserving maps, and we comment on…
Our capacity to process information depends on the computational power at our disposal. Information theory captures our ability to distinguish states or communicate messages when it is unconstrained with unrivaled beauty and elegance. For…
A well-known technique in estimating probabilities of rare events in general and in information theory in particular (used, e.g., in the sphere-packing bound), is that of finding a reference probability measure under which the event of…
We show how convergence to the Gumbel distribution in an extreme value setting can be understood in an information-theoretic sense. We introduce a new type of score function which behaves well under the maximum operation, and which implies…
In this paper, some new upper bounds for Kullback-Leibler divergence(KL-divergence) based on $L^1, L^2$ and $L^\infty$ norms of density functions are discussed. Our findings unveil that the convergence in KL-divergence sense sandwiches…
Boltzmann introduced in the 1870's a logarithmic measure for the connection between the thermodynamical entropy and the probabilities of the microscopic configurations of the system. His entropic functional for classical systems was…
B. Schumacher and M. Westmoreland have established a quantum analog of a well-known classical information theory result on a role of relative entropy as a measure of non-optimality in (classical) data compression. In this paper, we provide…