Related papers: On the Maximum Entropy of a Sum of Independent Dis…
The classical problem of maximizing the Shannon entropy of a sum of independent random variables supported on a finite alphabet is considered and settled in the ternary case. Namely, the following theorem is established: if…
We prove the Shepp--Olkin conjecture, which states that the entropy of the sum of independent Bernoulli random variables is concave in the parameters of the individual random variables. Our proof is a refinement of an argument previously…
Suppose that you have $n$ colours and $m$ mutually independent dice, each of which has $r$ sides. Each dice lands on any of its sides with equal probability. You may colour the sides of each die in any way you wish, but there is one…
We define a one-parameter family of entropies, each assigning a real number to any probability measure on a compact metric space (or, more generally, a compact Hausdorff space with a notion of similarity between points). These entropies…
This paper considers a variation of the full-information secretary problem where the random variables to be observed are independent but not necessary identically distributed. The main result is a sharp lower bound for the optimal win…
Let $X_1,\ldots,X_n$ be independent identically distributed random vectors in $\mathbb{R}^d$. We consider upper bounds on $\max_x \mathbb{P}(a_1X_1+\cdots+a_nX_n=x)$ under various restrictions on $X_i$ and the weights $a_i$. When…
We derive an asymptotic lower bound on the Shannon entropy $H$ of sums of $N$ arbitrary iid discrete random variables. The derived bound $H \geq \frac{r(X)}{2}\log(N) + {\it cst}$ is given in terms of the incommensurability rank $r(X)$ of…
We prove a Bernstein-type bound for the difference between the average of negative log-likelihoods of independent discrete random variables and the Shannon entropy, both defined on a countably infinite alphabet. The result holds for the…
Various lower bounds are established for the entropy of sums, products and their combinations. First, we derive a prime-field analogue of a version of the entropy power inequality established by Tao over torsion-free groups. Next, we prove…
This paper derives bounds for two omnipresent information theoretic measures, the Shannon entropy and its complementary dual, the extropy. Based on a large size data set from a logconcave model, the said bounds are obtained for the entropy…
Two new information-theoretic methods are introduced for establishing Poisson approximation inequalities. First, using only elementary information-theoretic techniques it is shown that, when $S_n=\sum_{i=1}^nX_i$ is the sum of the (possibly…
This paper considers the estimation of Shannon entropy for discrete distributions with countably infinite support. While minimax rates for finite-support distributions are established, infinite-support distributions present distinct…
Given a sequence of orthonormal polynomials on $\Bbb R$,$\{p_n\}_{n\geq 0}$, with $p_n$ of degree $n$, we define the discrete probability distribution $\Psi_n(x) = \left(\Psi_{n,1}(x), \dots \Psi_{n,n}(x) \right) $, with $\Psi_{n,j}(x) =…
Within a framework of utmost generality, we show that the entropy maximization procedure with linear constraints uniquely leads to the Shannon-Boltzmann-Gibbs entropy. Therefore, the use of this procedure with linear constraints should not…
This paper provides tight bounds on the R\'enyi entropy of a function of a discrete random variable with a finite number of possible values, where the considered function is not one-to-one. To that end, a tight lower bound on the R\'enyi…
We prove upper bounds on the rate, called "mixing rate", at which the von Neumann entropy of the expected density operator of a given ensemble of states changes under non-local unitary evolution. For an ensemble consisting of two states,…
Consider tossing a collection of coins, each fair or biased towards heads, and take the distribution of the total number of heads that result. It is natural to conjecture that this distribution should be 'more random' when each coin is…
The weak law of large numbers implies that, under mild assumptions on the source, the Renyi entropy per produced symbol converges (in probability) towards the Shannon entropy rate. This paper quantifies the speed of this convergence for…
We present novel bounds for estimating discrete probability distributions under the $\ell_\infty$ norm. These are nearly optimal in various precise senses, including a kind of instance-optimality. Our data-dependent convergence guarantees…
It is proven that a conjecture of Tao (2010) holds true for log-concave random variables on the integers: For every $n \geq 1$, if $X_1,\ldots,X_n$ are i.i.d. integer-valued, log-concave random variables, then $$ H(X_1+\cdots+X_{n+1}) \geq…