Related papers: An Effective Bernstein-type Bound on Shannon Entro…

We give estimates for the zero loci of Bernstein-Sato ideals. An upper bound is proved as a multivariate generalisation of the upper bound by Lichtin for the roots of Bernstein-Sato polynomials. The lower bounds generalise the fact that…

The classical Poisson theorem says that if $\xi_1,\xi_2,...$ are i.i.d. 0--1 Bernoulli random variables taking on 1 with probability $p_n\equiv \la/n$ then the sum $S_n=\sum_{i=1}^n\xi_i$ is asymptotically in $n$ Poisson distributed with…

Understanding the way in which random entities interact is of key interest in numerous scientific fields. This can range from a full characterization of the joint distribution to single scalar summary statistics. In this work we identify a…

We outline a general procedure on how to apply random positive linear operators in nonparametric estimation. As a consequence, we give explicit confidence bands and intervals for a distribution function $F$ concentrated on $[0,1]$ by means…

We view sequential design as a model selection problem to determine which new observation is expected to be the most informative, given the existing set of observations. For estimating a probability distribution on a bounded interval, we…

A classical random variable can be faithfully compressed into a sequence of bits with its expected length lies within one bit of Shannon entropy. We generalize this variable-length and faithful scenario to the general quantum source…

In this paper we develop a very general class of bivariate discrete distributions. The basic idea is very simple. The marginals are obtained by taking the random geometric sum of a baseline distribution function. The proposed class of…

We study the Shannon entropy of the cluster size distribution in classical as well as explosive percolation, in order to estimate the uncertainty in the sizes of randomly chosen clusters. At the critical point the cluster size distribution…

I consider the effect of a finite sample size on the entropy of a sample of independent events. I propose formula for entropy which satisfies Shannon's axioms, and which reduces to Shannon's entropy when sample size is infinite. I discuss…

One of the difficulties in calculating the capacity of certain Poisson channels is that H(lambda), the entropy of the Poisson distribution with mean lambda, is not available in a simple form. In this work we derive upper and lower bounds…

We prove semi-empirical concentration inequalities for random variables which are given as possibly nonlinear functions of independent random variables. These inequalities describe concentration of random variable in terms of the…

Using the theory of negative association for measures and the notion of random weak limits of sparse graphs, we establish the validity of the cavity method for counting spanning subgraphs subject to local constraints in asymptotically…

This monograph presents a unified treatment of single- and multi-user problems in Shannon's information theory where we depart from the requirement that the error probability decays asymptotically in the blocklength. Instead, the error…

We prove Bernstein-type matrix concentration inequalities for linear combinations with matrix coefficients of binary random variables satisfying certain $\ell_\infty$-independence assumptions, complementing recent results by Kaufman, Kyng…

We study the problem of generating a random variate $X$ from a finite discrete probability distribution $P$ using an entropy source of independent fair coin flips. A classic result from Knuth and Yao shows that the optimal expected number…

This paper develops an optimal Chernoff type bound for the probabilities of large deviations of sums $\sum_{k=1}^n f (X_k)$ where $f$ is a real-valued function and $(X_k)_{k \in \mathbb{Z}_{\ge 0}}$ is a finite state Markov chain with an…

We use the formalism of 'Maximum Principle of Shannon's Entropy' to derive the general power law distribution function, using what seems to be a reasonable physical assumption, namely, the demand of a constant mean "internal order"…

We show a direct relationship between the variance and the differential entropy for subclasses of symmetric and asymmetric unimodal distributions by providing an upper bound on variance in terms of entropy power. Combining this bound with…

We prove a Chernoff-type upper variance bound for the multinomial and the negative multinomial distribution. An application is also given.

We say that a random integer variable $X$ is monotone if the modulus of the characteristic function of $X$ is decreasing on $[0,\pi]$. This is the case for many commonly encountered variables, e.g., Bernoulli, Poisson and geometric random…