Related papers: Chain Rules for Smooth Min- and Max-Entropies
It is well known that a Shannon based definition of information entropy leads in the classical case to the Boltzmann entropy. It is tempting to regard the Von Neumann entropy as the corresponding quantum mechanical definition. But the…
In this paper, we present a new multi-scale information content calculation method based on Shannon information (and Shannon entropy). The original method described by Claude E. Shannon and based on the logarithm of the probability of…
The Shannon entropy is a fundamental measure for quantifying diversity and model complexity in fields such as information theory, ecology, and genetics. However, many existing studies assume that the number of species is known, an…
For noncomposite systems in classical and quantum domains, we obtain new inequalities such as the subadditivity and strong subadditivity conditions for Shannon entropies and information determined by the probability distributions and for…
We review subbadditivity properties of Shannon entropy, in particular, from the Shearer's inequality we derive the "infimum rule" for actions of amenable groups. We briefly discuss applicability of the "infimum formula" to actions of other…
Information inequalities govern the ultimate limitations in information theory and as such play an pivotal role in characterizing what values the entropy of multipartite states can take. Proving an information inequality, however, quickly…
The classical Maximum-Entropy Principle (MEP) based on Shannon entropy is widely used to construct least-biased probability distributions from partial information. However, the Shore-Johnson axioms that single out the Shannon functional…
We propose a new interpretation of measures of information and disorder by connecting these concepts to group theory in a new way. Entropy and group theory are connected here by their common relation to sets of permutations. A combinatorial…
We investigate the Shannon entropy of the total system and its subsystems, as well as the subsystem Shannon mutual information, in quasiparticle excited states of free bosonic and fermionic chains and the ferromagnetic phase of the spin-1/2…
We study an information-theoretic measure of uncertainty for quantum systems. It is the Shannon information $I$ of the phase space probability distribution $\la z | \rho | z \ra $, where $|z \ra $ are coherent states, and $\rho$ is the…
In the present paper we prove a family of tight upper and lower bounds for the Shannon entropy and von Neumann entropy based on the p-norms. This allows us to have an entropy estimate, a criterion for the finiteness of it and a bound on the…
In the 21st century, many of the crucial scientific and technical issues facing humanity can be understood as problems associated with understanding, modelling, and ultimately controlling complex systems: systems comprised of a large number…
Isomorph theory is one of the promising theories to understand the quasi-universal relationship between thermodynamic, dynamic and structural characteristics. Based on the hidden scale invariance of the inverse power law potentials, it…
This paper focuses on the extreme-value problem for Shannon entropy of the joint distribution with given marginals. It is proved that the minimum-entropy coupling must be of order-preserving, while the maximum-entropy coupling coincides…
Shannon entropy was defined for probability distributions and then its using was expanded to measure the uncertainty of knowledge for systems with complete information. In this article, it is proposed to extend the using of Shannon entropy…
The general idea of information entropy provided by C.E. Shannon "hangs over everything we do" and can be applied to a great variety of problems once the connection between a distribution and the quantities of interest is found. The Shannon…
Statements of Shannon's Noiseless Coding Theorem by various authors, including the original, are reviewed and clarified. Traditional statements of the theorem are often unclear as to when it applies. A new notation is introduced and the…
There is an asymptotic relationship between the multiplicative relations among multinomial coefficients and the (additive) recurrence property of Shannon entropy known as the chain rule. We show that both types of identities are…
We consider entanglement in the ground state of the XY spin model on infinite chain. We use von Neumann entropy of a sub-system as a measure of entanglement. The entropy of a large block of neighboring spins approaches a constant as the…
The weak law of large numbers implies that, under mild assumptions on the source, the Renyi entropy per produced symbol converges (in probability) towards the Shannon entropy rate. This paper quantifies the speed of this convergence for…