Related papers: A new entropy power inequality for integer-valued …
The theory of Extreme Physical Information (EPI) is used to deduce a probability density function (PDF) of a system that exhibits a power law tail. The computed PDF is useful to study and fit several observed distributions in complex…
We establish quantitative stability results for the entropy power inequality (EPI). Specifically, we show that if uniformly log-concave densities nearly saturate the EPI, then they must be close to Gaussian densities in the quadratic…
We introduce novel information-entropic variables -- a Point Divergence Gain (${\Omega}^{(l \rightarrow m)}_\alpha$), a Point Divergence Gain Entropy ($I_\alpha$), and a Point Divergence Gain Entropy Density ($P_\alpha$) -- which are…
Over the past few years, a family of interesting new inequalities for the entropies of sums and differences of random variables has been developed by Ruzsa, Tao and others, motivated by analogous results in additive combinatorics. The…
The asymptotic equipartition property (AEP) states that in the limit of a large number of independent and identically distributed (i.i.d.) random experiments, the output sequence is virtually certain to come from the typical set, each…
The estimation of information measures of continuous distributions based on samples is a fundamental problem in statistics and machine learning. In this paper, we analyze estimates of differential entropy in $K$-dimensional Euclidean space,…
This paper provides tight bounds on the R\'enyi entropy of a function of a discrete random variable with a finite number of possible values, where the considered function is not one-to-one. To that end, a tight lower bound on the R\'enyi…
We investigate the arithmetic-harmonic inequality (AHI) index, a bounded and scale-invariant measure of dispersion for positive random variables, defined through the interplay between the mean and its reciprocal. We derive analytical…
We study the increase in per-sample differential entropy rate of random sequences and processes after being passed through a non minimum-phase (NMP) discrete-time, linear time-invariant (LTI) filter G. For such filters and random processes,…
We establish a procedure to find the extremal density matrices for any finite Hamiltonian of a qudit system. These extremal density matrices provide an approximate description of the energy spectra of the Hamiltonian. In the case of…
We show how to determine the maximum and minimum possible values of one measure of entropy for a given value of another measure of entropy. These maximum and minimum values are obtained for two standard forms of probability distribution (or…
Entropy rate is a real valued functional on the space of discrete random sources which lacks a closed formula even for subclasses of sources which have intuitive parameterizations. A good way to overcome this problem is to examine its…
The entropy of a graph is an information-theoretic quantity which expresses the complexity of a graph \cite{DM1,M}. After Shannon introduced the definition of entropy to information and communication, many generalizations of the entropy…
We present a unifying approach to the study of entropies in Mathematics, such as measure entropy, topological entropy, algebraic entropy, set-theoretic entropy. We take into account discrete dynamical systems, that is, pairs $(X,T)$, where…
Maximum entropy estimation is of broad interest for inferring properties of systems across many different disciplines. In this work, we significantly extend a technique we previously introduced for estimating the maximum entropy of a set of…
This paper derives new bounds on the difference of the entropies of two discrete random variables in terms of the local and total variation distances between their probability mass functions. The derivation of the bounds relies on maximal…
Integral representations of quantum relative entropy, and of the directional second and higher order derivatives of von Neumann entropy, are established, and used to give simple proofs of fundamental, known data processing inequalities: the…
Nonequilibrium processes break time-reversal symmetry and generate entropy. Living systems are driven out-of-equilibrium at the microscopic level of molecular motors that exploit chemical potential gradients to transduce free energy to…
A strengthened version of the central limit theorem for discrete random variables is established, relying only on information-theoretic tools and elementary arguments. It is shown that the relative entropy between the standardised sum of…
For a random variable we can define a variational relationship with practical physical meaning as dI=dbar(x)-bar(dx), where I is called as uncertainty measurement. With the help of a generalized definition of expectation,…