Related papers: Average Entropy Functions
The notion of entropy appears in many fields and this paper is a survey about entropies in several branches of Mathematics. We are mainly concerned with the topological and the algebraic entropy in the context of continuous endomorphisms of…
We describe five types of results concerning information and concentration of discrete random variables, and relationships between them, motivated by their counterparts in the continuous case. The results we consider are information…
We consider the reduced density matrix of a large block of consecutive spins in the ground states of the XY spin chain on an infinite lattice. We derive the spectrum of the density matrix using the expression of the Renyi entropy in terms…
We introduce a taxonomy for partially coherent optical fields spanning multiple degrees of freedom (DoFs) based on the rank of the associated coherence matrix (the number of non-zero eigenvalues). When DoFs comprise two spatial modes and…
We consider a Gaussian statistical model whose parameter space is given by the variances of random variables. Underlying this model we identify networks by interpreting random variables as sitting on vertices and their correlations as…
Entropy and information can be considered dual: entropy is a measure of the subspace defined by the information constraining the given ambient space. Negative entropies, arising in na\"ive extensions of the definition of entropy from…
We express the entropy of a scalar field phi directly in terms of its spacetime correlation function W(x,y) = <phi(x) phi(y)>, assuming that the higher correlators are of "Gaussian" form. The resulting formula associates an entropy S(R) to…
The requirement that an entropy function be composable is key: it means that the entropy of a compound system can be calculated in terms of the entropy of its independent components. We prove that, under mild regularity assumptions, the…
The entanglement entropy of a subsystem $A$ of a quantum system is expressed, in the replica method, through analytic continuation with respect to n of the trace of the n-th power of the reduced density matrix $\tr\rho_A^n$. We study the…
All correlation measures, classical and quantum, must be monotonic under local operations. In this paper, we characterize monotonic formulas that are linear combinations of the von Neumann entropies associated with the quantum state of a…
It is not obvious how to extend Shannon's original information entropy to higher dimensions, and many different approaches have been tried. We replace the English text symbol sequence originally used to illustrate the theory by a discrete,…
This paper introduces a framework for modeling cyclical and feedback-driven information flow through a generalized family of entropy-modulated transformations called derangetropy functionals. Unlike scalar and static entropy measures such…
Shannon entropy in position ($S_{\rvec}$) and momentum ($S_{\pvec}$) spaces, along with their sum ($S_t$) are presented for unit-normalized densities of He, Li$^+$ and Be$^{2+}$ ions, spatially confined at the center of an impenetrable…
The problem of Shannon entropy estimation in countable infinite alphabets is addressed from the study and use of convergence results of the entropy functional, which is known to be discontinuous with respect to the total variation distance…
In this note a notion of generalized topological entropy for arbitrary subsets of the space of all sequences in a compact topological space is introduced. It is shown that for a continuous map on a compact space the generalized topological…
Information diagram and the I-measure are useful mnemonics where random variables are treated as sets, and entropy and mutual information are treated as a signed measure. Although the I-measure has been successful in machine proofs of…
We consider an isomorphism invariant for measure-preserving systems - types of generalized entropy convergence rates. We show the connections of this invariant with the types of Shannon entropy convergence rates. In the case when they…
Computations of entropy in thermodynamics rely on discreteness of the spectra of the subsystems. We argue that, for cases with continuous spectra (typically, radiation), there is a useful definition of entropy flow based on discretizing the…
In this note, we show that the relative entropy of an empirical distribution of $n$ samples drawn from a set of size $k$ with respect to the true underlying distribution is exponentially concentrated around its expectation, with central…
Shannon entropy, a cornerstone of information theory, statistical physics and inference methods, is uniquely identified by the Shannon-Khinchin or Shore-Johnson axioms. Generalizations of Shannon entropy, motivated by the study of…