Related papers: Symmetry in Shannon's Noiseless Coding Theorem

In this paper, we investigate the asymptotic stability of finite-dimensional stochastic integrable Hamiltonian systems via information entropy. Specifically, we establish the asymptotic vanishing of Shannon entropy difference (with…

Shannon's metric of "Entropy" of information is a foundational concept of information theory. This article is a primer for novices that presents an intuitive way of understanding, remembering, and/or reconstructing Shannon's Entropy metric…

The quantification of the complexity of networks is, today, a fundamental problem in the physics of complex systems. A possible roadmap to solve the problem is via extending key concepts of information theory to networks. In this paper we…

This paper derives bounds for two omnipresent information theoretic measures, the Shannon entropy and its complementary dual, the extropy. Based on a large size data set from a logconcave model, the said bounds are obtained for the entropy…

We present a quantum information theory that allows for a consistent description of entanglement. It parallels classical (Shannon) information theory but is based entirely on density matrices (rather than probability distributions) for the…

Complementarity relations between various characterizations of a probability distribution are at the core of information theory. In particular, lower and upper bounds for the entropic function are of great importance. In applied topics, we…

Shannon entropy was defined for probability distributions and then its using was expanded to measure the uncertainty of knowledge for systems with complete information. In this article, it is proposed to extend the using of Shannon entropy…

It is pointed out that the case for Shannon entropy and von Neumann entropy, as measures of uncertainty in quantum mechanics, is not as bleak as suggested in quant-ph/0006087. The main argument of the latter is based on one particular…

We show that an information-theoretic property of Shannon's entropy power, known as concavity of entropy power, can be fruitfully employed to prove inequalities in sharp form. In particular, the concavity of entropy power implies the…

The problem of Shannon entropy estimation in countable infinite alphabets is addressed from the study and use of convergence results of the entropy functional, which is known to be discontinuous with respect to the total variation distance…

Whereas Shannon entropy is related to the growth rate of multinomial coefficients, we show that the quadratic entropy (Tsallis 2-entropy) is connected to their $q$-deformation; when $q$ is a prime power, these $q$-multinomial coefficients…

A system of interacting qubits can be viewed as a non-i.i.d quantum information source. A possible model of such a source is provided by a quantum spin system, in which spin-1/2 particles located at sites of a lattice interact with each…

We define a large class of quantum sources and prove a quantum analog of the asymptotic equipartition property. Our proof relies on using local measurements on the quantum source to obtain an associated classical source. The classical…

A tight information-theoretic measurement uncertainty relation is experimentally tested with neutron spin-1/2 qubits. The noise associated to the measurement of an observable is defined via conditional Shannon entropies and a tradeoff…

"Bounds on information combining" are entropic inequalities that determine how the information (entropy) of a set of random variables can change when these are combined in certain prescribed ways. Such bounds play an important role in…

I consider the effect of a finite sample size on the entropy of a sample of independent events. I propose formula for entropy which satisfies Shannon's axioms, and which reduces to Shannon's entropy when sample size is infinite. I discuss…

This thesis consolidates, improves and extends the smooth entropy framework for non-asymptotic information theory and cryptography. We investigate the conditional min- and max-entropy for quantum states, generalizations of classical R\'enyi…

We review with a tutorial scope the information theory foundations of quantum statistical physics. Only a small proportion of the variables that characterize a system at the microscopic scale can be controlled, for both practical and…

A thermodynamic framework for asymptotic inference is developed in which sample size and parameter variance define a state space. Within this description, Shannon information plays the role of entropy, and an integrating factor organizes…

In this paper we discuss about the validity of the Shannon entropy functional in connection with the correct Gibbs-Hertz probability distribution function. We show that there is no contradiction in using the Shannon-Gibbs functional and…