Related papers: Concavity of entropy under thinning

The concept of Entropy plays a key role in Information Theory, Statistics, and Machine Learning.This paper introduces a new entropy measure, called the t-entropy, which exploits the concavity of the inverse-tan function. We analytically…

We embark on a systematic study of the $(k+1)$-th derivative of $x^{k-r}H(x^r)$, where $H(x):=-x\log x-(1-x)\log(1-x)$ is the binary entropy and $k>r\geq 1$ are integers. Our motivation is the conjectural entropy inequality $\alpha_k…

Entropy and other fundamental quantities of information theory are customarily expressed and manipulated as functions of probabilities. Here we study the entropy H and subentropy Q as functions of the elementary symmetric polynomials in the…

We show that there is equality in Shannon's Entropy Power Inequality (EPI) if and only if the random variables involved are Gaussian, assuming nothing beyond the existence of differential entropies. This is done by justifying de Bruijn's…

Let $\cal{N}=\{1,\cdots,n\}$. The entropy function $\bf h$ of a set of $n$ discrete random variables $\{X_i:i\in\cal N\}$ is a $2^n$-dimensional vector whose entries are ${\bf{h}}({\cal{A}})\triangleq H(X_{\cal{A}}),\cal{A}\subset{\cal N}…

The classical problem of maximizing the Shannon entropy of a sum of independent random variables supported on a finite alphabet is considered and settled in the ternary case. Namely, the following theorem is established: if…

To discuss the existence and uniqueness of proper scoring rules one needs to extend the associated entropy functions as sublinear functions to the conic hull of the prediction set. In some natural function spaces, such as the Lebesgue…

The paper is concerned with proving the equivalence of convexity or concavity properties of thermodynamic functions, such as energy and entropy, depending on different sets of variables. These variables are the basic thermodynamic state…

In this article, we present a new subadditivity behavior of convex and concave functions, when applied to Hilbert space operators. For example, under suitable assumptions on the spectrum of the positive operators $A$ and $B$, we prove that…

A short and elementary proof of the joint convexity of relative entropy is presented, using nothing beyond linear algebra. The key ingredients are an easily verified integral representation and the strategy used to prove the Cauchy-Schwarz…

We show, using strong subadditivity and Lorentz covariance, that in three dimensional space-time the entanglement entropy of a circle is a concave function. This implies the decrease of the coefficient of the area term and the increase of…

We have presented a new axiomatic derivation of Shannon Entropy for a discrete probability distribution on the basis of the postulates of additivity and concavity of the entropy function.We have then modified shannon entropy to take account…

The notion of ordinal concavity of utility functions has recently been considered by Hafalir, Kojima, Yenmez, and Yokote in economics while there exist earlier related works in discrete optimization and operations research. In the present…

We present estimators for entropy and other functions of a discrete probability distribution when the data is a finite sample drawn from that probability distribution. In particular, for the case when the probability distribution is a joint…

Let $X$ be an arbitrary continuous random variable and $Z$ be an independent Gaussian random variable with zero mean and unit variance. For $t~>~0$, Costa proved that $e^{2h(X+\sqrt{t}Z)}$ is concave in $t$, where the proof hinged on the…

Convexity properties of the entropy along displacement interpolations are crucial in the Lott-Sturm-Villani theory of lower bounded curvature of geodesic measure spaces. As discrete spaces fail to be geodesic, an alternate analogous theory…

The problem of Shannon entropy estimation in countable infinite alphabets is addressed from the study and use of convergence results of the entropy functional, which is known to be discontinuous with respect to the total variation distance…

We study the evolution of Tsallis entropy along the heat flow and establish its concavity in arbitrary dimensions. Extending prior results that were restricted to the one-dimensional setting, we prove that the Tsallis entropy is concave in…

We prove stability estimates for the Shannon-Stam inequality (also known as the entropy-power inequality) for log-concave random vectors in terms of entropy and transportation distance. In particular, we give the first stability estimate…

It was recently shown that estimating the Shannon entropy $H({\rm p})$ of a discrete $k$-symbol distribution ${\rm p}$ requires $\Theta(k/\log k)$ samples, a number that grows near-linearly in the support size. In many applications $H({\rm…