Related papers: Simple inequalities for weighted entropies

Relations between Shannon entropy and Renyi entropies of integer order are discussed. For any N-point discrete probability distribution for which the Renyi entropies of order two and three are known, we provide an lower and an upper bound…

Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension…

The expression for entropy sometimes appears mysterious - as it often is asserted without justification. This short manuscript contains a discussion of the underlying assumptions behind entropy as well as simple derivation of this…

The derivation of Bell inequalities in terms of quantum statistical (thermodynamic) entropies is considered. Inequalities of the Wigner form are derived but shown to be extremely limiting in their applicability due to the nature of the…

Various lower bounds are established for the entropy of sums, products and their combinations. First, we derive a prime-field analogue of a version of the entropy power inequality established by Tao over torsion-free groups. Next, we prove…

We show that Shannon's entropy--power inequality admits a strengthened version in the case in which the densities are log-concave. In such a case, in fact, one can extend the Blachman--Stam argument to obtain a sharp inequality for the…

In this paper we have shall generalize Shearer's entropy inequality and its recent extensions by Madiman and Tetali, and shall apply projection inequalities to deduce extensions of some of the inequalities concerning sums of sets of…

Shannon's metric of "Entropy" of information is a foundational concept of information theory. This article is a primer for novices that presents an intuitive way of understanding, remembering, and/or reconstructing Shannon's Entropy metric…

We produce a series of results extending information-theoretical inequalities (discussed by Dembo--Cover--Thomas in 1989-1991) to a weighted version of entropy. The resulting inequalities involve the Gaussian weighted entropy; they imply a…

In this paper we remark that Shannon entropy can be expressed as a function of the self-information (i.e. the logarithm) and the inverse of the Lambert $W$ function. It means that we consider that Shannon entropy has the trace form: $-k…

Entropic uncertainty relations are interesting in their own rights as well as for a lot of applications. Keeping this in mind, we try to make the corresponding inequalities as tight as possible. The use of parametrized entropies also allows…

We offer an overview of the specification property, its relatives and their consequences. We examine relations between specification-like properties and such notions as: mixing, entropy, the structure of the simplex of invariant measures,…

In the past over two decades, very fruitful results have been obtained in information theory in the study of the Shannon entropy. This study has led to the discovery of a new class of constraints on the Shannon entropy called…

The Shannon entropy, one of the cornerstones of information theory, is widely used in physics, particularly in statistical mechanics. Yet its characterization and connection to physics remain vague, leaving ample room for misconceptions and…

We show that an inequality related to Newton's inequality provides one more relation between skewness and kurtosis. This also gives simple and alternative proofs of the bounds for skewness and kurtosis.

Generalized entropies are studied as Lyapunov functions for the Master equation (Markov chains). Three basic properties of these Lyapunov functions are taken into consideration: universality (independence of the kinetic coefficients),…

We show that an information-theoretic property of Shannon's entropy power, known as concavity of entropy power, can be fruitfully employed to prove inequalities in sharp form. In particular, the concavity of entropy power implies the…

We obtain a mathematically simple characterization of all functionals coinciding with the von Neumann reduced entropy on pure states based on the Khinchin-Faddeev axiomatization of Shannon entropy and give a physical interpretation of the…

Entropy has emerged as a dynamic, interdisciplinary, and widely accepted quantitative measure of uncertainty across different disciplines. A unified understanding of entropy measures, supported by a detailed review of their theoretical…

The entropy power inequality for independent random vectors is a foundational result of information theory, with deep connections to probability and geometric functional analysis. Several extensions of the entropy power inequality have been…