Related papers: An Entropy Power Inequality for Discrete Random Va…
We prove the following type of discrete entropy monotonicity for sums of isotropic, log-concave, independent and identically distributed random vectors $X_1,\dots,X_{n+1}$ on $\mathbb{Z}^d$: $$ H(X_1+\cdots+X_{n+1}) \geq H(X_1+\cdots+X_{n})…
This paper studies the problem of estimating the differential entropy $h(S+Z)$, where $S$ and $Z$ are independent $d$-dimensional random variables with $Z\sim\mathcal{N}(0,\sigma^2 \mathrm{I}_d)$. The distribution of $S$ is unknown, but $n$…
It is proven that a conjecture of Tao (2010) holds true for log-concave random variables on the integers: For every $n \geq 1$, if $X_1,\ldots,X_n$ are i.i.d. integer-valued, log-concave random variables, then $$ H(X_1+\cdots+X_{n+1}) \geq…
Within generalized random energy models, we study the effects of energy discreteness and of entropy extensivity in the low temperature phase. At zero temperature, discreteness of the energy induces replica symmetry breaking, in contrast to…
We prove an exponential decay concentration inequality to bound the tail probability of the difference between the log-likelihood of discrete random variables on a finite alphabet and the negative entropy. The concentration bound we derive…
In this paper we consider a weighted version of one dimensional discrete Hardy's Inequality on half-line with power weights of the form $n^\alpha$. Namely we consider: \begin{equation} \sum_{n=1}^\infty |u(n)-u(n-1)|^2 n^\alpha \geq…
Reliable data-driven estimation of Shannon entropy from small data sets, where the number of examples is potentially smaller than the number of possible outcomes, is a critical matter in several applications. In this paper, we introduce a…
We improve the classical discrete Hardy inequality for $ 1<p<\infty $ for functions on the natural numbers. For integer values of $ p $ the Hardy weight is an absolutely monotonic function.
Entropy and information can be considered dual: entropy is a measure of the subspace defined by the information constraining the given ambient space. Negative entropies, arising in na\"ive extensions of the definition of entropy from…
For a closed-loop control system with a digital channel between the sensor and the controller, the notion of invariance entropy quantifies the smallest average rate of information transmission above which a given compact subset of the state…
We provide a condition under which a version of Shannon's Entropy Power Inequality will hold for dependent variables. We provide information inequalities extending those found in the independent case.
The time decay of fully discrete finite-volume approximations of porous-medium and fast-diffusion equations with Neumann or periodic boundary conditions is proved in the entropy sense. The algebraic or exponential decay rates are computed…
We prove a lower estimate on the increase in entropy when two copies of a conditional random variable $X | Y$, with $X$ supported on $\mathbb{Z}_q=\{0,1,\dots,q-1\}$ for prime $q$, are summed modulo $q$. Specifically, given two i.i.d copies…
Shannon's entropy power inequality (EPI) can be viewed as a statement of concavity of an entropic function of a continuous random variable under a scaled addition rule: $$f(\sqrt{a}\,X + \sqrt{1-a}\,Y) \ge a f(X) + (1-a) f(Y) \quad \forall…
This paper explores some applications of a two-moment inequality for the integral of the $r$-th power of a function, where $0 < r< 1$. The first contribution is an upper bound on the R\'{e}nyi entropy of a random vector in terms of the two…
Let $S(\rho)=- Tr (\rho \log\rho)$ be the von Neumann entropy of an $N$-dimensional quantum state $\rho$ and $e_2(\rho)$ the second elementary symmetric polynomial of the eigenvalues of $\rho$. We prove the inequality $S(\rho) \le c(N)…
We propose utilizing entropy as a diagnostic tool to distinguish between constant and dynamical dark energy models. Entropy, a measure of the system's disorder or information content, captures the complexity and evolution of the universe.…
Entropy, its production, and its change in a dynamical system can be understood from either a fully stochastic dynamic description or from a deterministic dynamics exhibiting chaotic behavior. By taking the former approach based on the…
We consider discrete and continuous representations of a thermodynamic process in which a random walker (e.g. a molecular motor on a molecular track) uses a periodically pumped energy (work) to pass $N$ sites and move energetically downhill…
In this article we study weighted sums of $n$ i.i.d. Gamma($\alpha$) random variables with nonnegative weights. We show that for $n \geq 1/\alpha$ the sum with equal coefficients maximizes differential entropy when variance is fixed. As a…