Related papers: Simple inequalities for weighted entropies
The routine definitions of both entropy, and differential entropy show inconsistencies that make them not reciprocally coherent. We propose a few possible modifications of these quantities so that 1) they no longer show incongruities, 2)…
We consider an isomorphism invariant for measure-preserving systems - types of generalized entropy convergence rates. We show the connections of this invariant with the types of Shannon entropy convergence rates. In the case when they…
I consider the effect of a finite sample size on the entropy of a sample of independent events. I propose formula for entropy which satisfies Shannon's axioms, and which reduces to Shannon's entropy when sample size is infinite. I discuss…
We study conditional linear information inequalities, i.e., linear inequalities for Shannon entropy that hold for distributions whose entropies meet some linear constraints. We prove that some conditional information inequalities cannot be…
Aim of this short note is to study Shannon's entropy power along entropic interpolations, thus generalizing Costa's concavity theorem. We shall provide two proofs of independent interest: the former by $\Gamma$-calculus, hence applicable to…
We study an entropy measure for quantum systems that generalizes the von Neumann entropy as well as its classical counterpart, the Gibbs or Shannon entropy. The entropy measure is based on hypothesis testing and has an elegant formulation…
This paper addresses the correspondence between linear inequalities of Shannon entropy and differential entropy for sums of independent group-valued random variables. We show that any balanced (with the sum of coefficients being zero)…
In this article, we discuss the problem of establishing relations between information measures assessed for network structures. Two types of entropy based measures namely, the Shannon entropy and its generalization, the R\'{e}nyi entropy…
We consider the Shannon-Khinchin axiomatic systems for the characterization of generalized entropies such as Sharma-Mital and Frank-Daffertshofer entropy. We provide the generalization of Shannon-Khinchin axioms and give the corresponding…
A new concept named nonsymmetric entropy which generalizes the concepts of Boltzman's entropy and shannon's entropy, was introduced. Maximal nonsymmetric entropy principle was proven. Some important distribution laws were derived naturally…
We consider the problem of finite sample corrections for entropy estimation. New estimates of the Shannon entropy are proposed and their systematic error (the bias) is computed analytically. We find that our results cover correction…
R\'enyi entropy of order \alpha is a general measure of entropy. In this paper we derive estimations for the R\'enyi entropy of the mixture of sources in terms of the entropy of the single sources. These relations allow to compute the…
New families of Fisher information and entropy power inequalities for sums of independent random variables are presented. These inequalities relate the information in the sum of $n$ independent random variables to the information contained…
We introduce unbiased estimators for the Shannon entropy and the class number, in the situation that we are able to take sequences of independent samples of arbitrary length.
Using known entropic and information inequalities new inequalities for some classical polynomials are obtained. Examples of Jacobi and Legendre polynomials are considered.
Yet another simple proof of the entropy power inequality is given, which avoids both the integration over a path of Gaussian perturbation and the use of Young's inequality with sharp constant or R\'enyi entropies. The proof is based on a…
We present a new class of estimators of Shannon entropy for severely undersampled discrete distributions. It is based on a generalization of an estimator proposed by T. Schuermann, which itself is a generalization of an estimator proposed…
We have presented a new axiomatic derivation of Shannon Entropy for a discrete probability distribution on the basis of the postulates of additivity and concavity of the entropy function.We have then modified shannon entropy to take account…
Uncertainty relations provide constraints on how well the outcomes of incompatible measurements can be predicted, and, as well as being fundamental to our understanding of quantum theory, they have practical applications such as for…
We consider a probability distribution depending on a real parameter $x$. As functions of $x$, the R\'enyi entropy and the Tsallis entropy can be expressed in terms of the associated index of coincidence $S(x)$. We establish recurrence…