Related papers: Chain Rules for Smooth Min- and Max-Entropies
The notion of Shannon entropy is crucial for the theory of classical information. In quantum information theory, an analogous key role is played by the von Neumann entropy: quantum information processing is closely related to entropy…
Thermodynamic entropy, as defined by Clausius, characterizes macroscopic observations of a system based on phenomenological quantities such as temperature and heat. In contrast, information-theoretic entropy, introduced by Shannon, is a…
We normalize the combinatorial Laplacian of a graph by the degree sum, look at its eigenvalues as a probability distribution and then study its Shannon entropy. Equivalently, we represent a graph with a quantum mechanical state and study…
Two new relative entropy quantities, called the min- and max-relative entropies, are introduced and their properties are investigated. The well-known min- and max- entropies, introduced by Renner, are obtained from these. We define a new…
One-shot information theory entertains a plethora of entropic quantities, such as the smooth max-divergence, hypothesis testing divergence and information spectrum divergence, that characterize various operational tasks and are used to…
In information theory, one major goal is to find useful functions that summarize the amount of information contained in the interaction of several random variables. Specifically, one can ask how the classical Shannon entropy, mutual…
We study how the Shannon entropy of sequences produced by an information source converges to the source's entropy rate. We synthesize several phenomenological approaches to applying information theoretic measures of randomness and memory to…
The entropy accumulation theorem states that the smooth min-entropy of an $n$-partite system $A = (A_1, \ldots, A_n)$ is lower-bounded by the sum of the von Neumann entropies of suitably chosen conditional states up to corrections that are…
The Renyi entropies constitute a family of information measures that generalizes the well-known Shannon entropy, inheriting many of its properties. They appear in the form of unconditional and conditional entropies, relative entropies or…
Bounds on information combining are a fundamental tool in coding theory, in particular when analyzing polar codes and belief propagation. They usually bound the evolution of random variables with respect to their Shannon entropy. In recent…
We review with a tutorial scope the information theory foundations of quantum statistical physics. Only a small proportion of the variables that characterize a system at the microscopic scale can be controlled, for both practical and…
We characterize information as risk reduction between knowledge states represented by partitions of the underlying probability space. Entropy corresponds to risk reduction from no (or partial) knowledge to full knowledge about a random…
Relative entropy is the standard measure of distinguishability in classical and quantum information theory. In the classical case, its loss under channels admits an exact chain rule, while in the quantum case only asymptotic, regularized…
This article consists of a very short introduction to classical and quantum information theory. Basic properties of the classical Shannon entropy and the quantum von Neumann entropy are described, along with related concepts such as…
The primary entropic measures for quantum states are additive under the tensor product. In the analysis of quantum information processing tasks, the minimum entropy of a set of states, e.g., the minimum output entropy of a channel, often…
Von Neumann entropy (VNE) is a fundamental quantity in quantum information theory and has recently been adopted in machine learning as a spectral measure of diversity for kernel matrices and kernel covariance operators. While maximizing VNE…
Relations between Shannon entropy and Renyi entropies of integer order are discussed. For any N-point discrete probability distribution for which the Renyi entropies of order two and three are known, we provide an lower and an upper bound…
We derive a novel chain rule for a family of channel conditional entropies, covering von Neumann and sandwiched R\'{e}nyi entropies. In the process, we show that these channel conditional entropies are equal to their regularized version,…
The Shannon entropy of a collection of random variables is subject to a number of constraints, the best-known examples being monotonicity and strong subadditivity. It remains an open question to decide which of these "laws of information…
An "entropy increasing to the maximum" result analogous to the entropic central limit theorem (Barron 1986; Artstein et al. 2004) is obtained in the discrete setting. This involves the thinning operation and a Poisson limit. Monotonic…