Related papers: Approximate Discrete Entropy Monotonicity for Log-…

Several conjectures concern the lower bound for the differential entropy $H(X_t)$ of an $n$-dimensional random vector $X_t$ introduced by Costa. Cheng and Geng conjectured that $H(X_t)$ is completely monotone, that is, $C_1(m,n):…

Let $\lambda$ denote the Liouville function. A well known conjecture of Chowla asserts that for any distinct natural numbers $h_1,\dots,h_k$, one has $\sum_{1 \leq n \leq X} \lambda(n+h_1) \dotsm \lambda(n+h_k) = o(X)$ as $X \to \infty$.…

We show that $h_\infty(X+Y)\leq h_\infty(Z+W)$, where $X, Y$ are independent log-concave random variables, and $Z, W$ are exponential random variables having the same respective $\infty$-R\'enyi entropies. Analogs for integer-valued…

Given ergodic p-invariant measures {\mu_i} on the 1-torus T=R/Z, we give a sharp condition on their entropies, guaranteeing that the entropy of the convolution \muon converges to \log p. We also prove a variant of this result for joinings…

Given a sequence of orthonormal polynomials on $\Bbb R$,$\{p_n\}_{n\geq 0}$, with $p_n$ of degree $n$, we define the discrete probability distribution $\Psi_n(x) = \left(\Psi_{n,1}(x), \dots \Psi_{n,n}(x) \right) $, with $\Psi_{n,j}(x) =…

A lower bound on the probability $P(0<X<\delta)$ for all real $\delta>0$ and all random variables $X$ with log-concave p.d.f.'s such that $EX=0$ and $EX^2=1$ is obtained.

Given a sequence $(X_n)$ of symmetrical random variables taking values in a Hilbert space, an interesting open problem is to determine the conditions under which the series $\sum_{n=1}^\infty X_n$ is almost surely convergent. For…

The problem of Shannon entropy estimation in countable infinite alphabets is addressed from the study and use of convergence results of the entropy functional, which is known to be discontinuous with respect to the total variation distance…

We prove a free probability analog of a result of Artstein-Bally-Barthez-Naor. In particualar we prove that if X_{1},X_{2},... are freely independent identically distributed random variables, then the free entropy…

We prove a tight uniform continuity bound for the conditional Shannon entropy of discrete finitely supported random variables in terms of total variation distance.

A strengthened version of the central limit theorem for discrete random variables is established, relying only on information-theoretic tools and elementary arguments. It is shown that the relative entropy between the standardised sum of…

We extend Hoggar's theorem that the sum of two independent discrete-valued log-concave random variables is itself log-concave. We introduce conditions under which the result still holds for dependent variables. We argue that these…

We prove a variety of new and refined uniform continuity bounds for entropies of both classical random variables on an infinite state space and of quantum states of infinite-dimensional systems. We obtain the first tight continuity estimate…

This paper considers the estimation of Shannon entropy for discrete distributions with countably infinite support. While minimax rates for finite-support distributions are established, infinite-support distributions present distinct…

This paper studies the problem of estimating the differential entropy $h(S+Z)$, where $S$ and $Z$ are independent $d$-dimensional random variables with $Z\sim\mathcal{N}(0,\sigma^2 \mathrm{I}_d)$. The distribution of $S$ is unknown, but $n$…

It is well known that the entropy $H(X)$ of a discrete random variable $X$ is always greater than or equal to the entropy $H(f(X))$ of a function $f$ of $X$, with equality if and only if $f$ is one-to-one. In this paper, we give tight…

Let $S_n$ be the sum of independent random variables with distribution $F$. Under the assumption that $-\log(1-F(x))$ is slowly varying, conditions for $$ \lim_{n\to\infty}\sup_{s\ge t_n}\left|{P[S_n>s]\over n(1-F(s))}-1\right| =0 $$ are…

Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension…

Let $\mathsf{N}_{\rm d}\left[X\right]=\frac{1}{2\pi {\rm e}}{\rm e}^{2\mathsf{H}\left[X\right]}$ denote the entropy power of the discrete random variable $X$ where $\mathsf{H}\left[X\right]$ denotes the discrete entropy of $X$. In this…

We embark on a systematic study of the $(k+1)$-th derivative of $x^{k-r}H(x^r)$, where $H(x):=-x\log x-(1-x)\log(1-x)$ is the binary entropy and $k>r\geq 1$ are integers. Our motivation is the conjectural entropy inequality $\alpha_k…