Related papers: $H(X)$ vs. $H(f(X))$

It is known that if a 2-universal hash function $H$ is applied to elements of a {\em block source} $(X_1,...,X_T)$, where each item $X_i$ has enough min-entropy conditioned on the previous items, then the output distribution…

Given a knowledge base KB containing first-order and statistical facts, we consider a principled method, called the random-worlds method, for computing a degree of belief that some formula Phi holds given KB. If we are reasoning about a…

Pointwise H\"older exponents describe the degree of regularity of a function near a point. For a function $f:\mathbb{R}\to\mathbb{R}$, a number $\alpha>0$ and a point $t_0\in\mathbb{R}$, write $f\in C^\alpha(t_0)$ if there is a constant $C$…

Shannon entropy is widely used to quantify the uncertainty of discrete random variables. But when normalized to the unit interval, as is often done in practice, it no longer conveys the alphabet sizes of the random variables being studied.…

We establish Hoeffding-type concentration inequalities for the low and high tail bounds of sums of exchangeable random variables. Our results exhibit an anti-symmetry in such tail bounds due to the assumption of exchangeability, a…

We introduce the problem of \emph{entropy equivalence testing} for probability distributions, a relaxation of the well-studied closeness testing problem, where the distribution testing algorithm is now only required to distinguish, given…

The well-known Bennett-Hoeffding bound for sums of independent random variables is refined, by taking into account truncated third moments, and at that also improved by using, instead of the class of all increasing exponential functions,…

We consider families of finite quantum graphs of increasing size and we are interested in how eigenfunctions are distributed over the graph. As a measure for the distribution of an eigenfunction on a graph we introduce the entropy, it has…

We prove a general lower bound of quantum decision tree complexity in terms of some entropy notion. We regard the computation as a communication process in which the oracle and the computer exchange several rounds of messages, each round…

In this work, we revisit the one- and two-sample testing problems: binary hypothesis testing in which one or both distributions are unknown. For the one-sample test, we provide a more streamlined proof of the asymptotic optimality of…

An index of uniformity is developed as an alternative to the maximum-entropy principle for selecting continuous, differentiable probability distributions $\mathcal{P}$ subject to constraints $C$. The uniformity index developed in this paper…

We extend a result of Lyons (2016) from fractional tiling of finite graphs to a version for infinite random graphs. The most general result is as follows. Let $\bf P$ be a unimodular probability measure on rooted networks $(G, o)$ with…

Let $X$ be a random variable distributed according to the binomial distribution with parameters $n$ and $p$. It is shown that $P(X>EX)\ge1/4$ if $1>p\ge c/n$, where $c:=\ln(4/3)$, the best possible constant factor.

For the power-law quantum wave packet in configuration space, the variance of the position observable may be divergent. Accordingly, the information-entropic formulation of the uncertainty principle becomes more appropriate than the…

Shearer's inequality bounds the sum of joint entropies of random variables in terms of the total joint entropy. We give another lower bound for the same sum in terms of the individual entropies when the variables are functions of…

We investigate the behavior of the nonparametric maximum likelihood estimator $\hat{f}_n$ for a decreasing density $f$ near the boundaries of the support of $f$. We establish the limiting distribution of $\hat{f}_n(n^{-\alpha})$, where we…

Hoeffding's Inequality provides the maximum probability that a series of n draws from a bounded random variable differ from the variable's true expectation u by more than given tolerance t. The random variable is typically the error rate of…

We provide a lower bound on the probability that a binomial random variable is exceeding its mean. Our proof employs estimates on the mean absolute deviation and the tail conditional expectation of binomial random variables.

Here we present an analytic approximation for the entropy of floating-point numbers, along with bounds on the error of this approximation. It is well-known that the differential entropy is tightly linked to the discrete entropy of a…

The principle of maximum entropy provides a useful method for inferring statistical mechanics models from observations in correlated systems, and is widely used in a variety of fields where accurate data are available. While the assumptions…