Related papers: $H(X)$ vs. $H(f(X))$

Jensen's inequality is ubiquitous in measure and probability theory, statistics, machine learning, information theory and many other areas of mathematics and data science. It states that, for any convex function $f\colon K \to \mathbb{R}$…

Given an SFT $\Sigma$ and a finite set $S$ of finite words, let $\Sigma\langle S\rangle$ denote the subshift of $\Sigma$ that avoids $S$. We establish a general criterion under which we can bound the entropy perturbation…

Within the task of collaborative filtering two challenges for computing conditional probabilities exist. First, the amount of training data available is typically sparse with respect to the size of the domain. Thus, support for higher-order…

How low can the joint entropy of $n$ $d$-wise independent (for $d\ge2$) discrete random variables be, subject to given constraints on the individual distributions (say, no value may be taken by a variable with probability greater than $p$,…

We investigate the problem of testing the equivalence between two discrete histograms. A {\em $k$-histogram} over $[n]$ is a probability distribution that is piecewise constant over some set of $k$ intervals over $[n]$. Histograms have been…

For $0<\alpha<1$ let $V(\alpha)$ denote the supremum of the numbers $v$ such that every $\alpha$-H\"older continuous function is of bounded variation on a set of Hausdorff dimension $v$. Kahane and Katznelson (2009) proved the estimate $1/2…

Let $(X,T)$ be a topological dynamical system. We define the measure-theoretical lower and upper entropies $\underline{h}_\mu(T)$, $\bar{h}_\mu(T)$ for any $\mu\in M(X)$, where $M(X)$ denotes the collection of all Borel probability measures…

Fix a graph $H$ and some $p\in (0,1)$, and let $X_H$ be the number of copies of $H$ in a random graph $G(n,p)$. Random variables of this form have been intensively studied since the foundational work of Erd\H{o}s and R\'{e}nyi. There has…

The uncertainty principle is a fundamental principle in quantum physics. It implies that the measurement outcomes of two incompatible observables can not be predicted simultaneously. In quantum information theory, this principle can be…

Topological entropy is a widely studied indicator of chaos in topological dynamics. Here we give a generalized definition of topological entropy which may be applied to set-valued functions. We demonstrate that some of the well-known…

This paper derives a lower bound on the spacing between adjacent zeros of the confluent hypergeometric function $\Phi(a,b,z)$ when $a$ is variable and $(b,z) \in \mathbb{R}^+$ are known and fixed. Monotonicity of the bound is established,…

This article presents the calculation of the entropy of a system with Zipfian distribution and shows that a communication system tends to present an exponent value close to one, but still greater than one, so that it might maximize entropy…

The problem of determining the joint probability distributions for correlated random variables with pre-specified marginals is considered. When the joint distribution satisfying all the required conditions is not unique, the "most unbiased"…

Fox's H-function provide a unified and elegant framework to tackle several physical phenomena. We solve the space fractional diffusion equation on the real line equipped with a delta distribution initial condition and identify the…

Let $ X_1, \ldots, X_n $ be independent random variables taking values in the alphabet $ \{0, 1, \ldots, r\} $, and $ S_n = \sum_{i = 1}^n X_i $. The Shepp--Olkin theorem states that, in the binary case ($ r = 1 $), the Shannon entropy of $…

We derive a tight lower bound on equivocation (conditional entropy), or equivalently a tight upper bound on mutual information between a signal variable and channel outputs. The bound is in terms of the joint distribution of the signals and…

Entropic uncertainty relations in a finite dimensional Hilbert space are investigated. Making use of the majorization technique we derive explicit lower bounds for the sum of R\'enyi entropies describing probability distributions associated…

We show a general phenomenon of the constrained functional value for densities satisfying general convexity conditions, which generalizes the observation in Bobkov and Madiman (2011) that the entropy per coordinate in a log-concave random…

We provide lower and upper bounds on the information transmission capacity of one single use of a classical-quantum channel. The lower bound is expressed in terms of the Hoeffding capacity, that we define similarly to the Holevo capacity,…

Under the Riemann hypothesis, we use the distribution of zeros of the zeta function to get a lower bound for the maximum of some derivative of Hardy's function.