Related papers: Entropy Bounds for Discrete Random Variables via M…
We present novel bounds for estimating discrete probability distributions under the $\ell_\infty$ norm. These are nearly optimal in various precise senses, including a kind of instance-optimality. Our data-dependent convergence guarantees…
Using entropic inequalities from information theory, we provide new bounds on the total variation and 2-Wasserstein distances between a conditionally Gaussian law and a Gaussian law with invertible covariance matrix. We apply our results to…
In this article, I give a definition of topological entropy for random dynamical systems associated to an infinite countable discrete amenable group action. I obtain a variational principle between the topological entropy and measurable…
Sums of of 1-dependent integer-valued random variables are approximated by compound Poisson, negative binomial and Binomial distributions and signed compound Poisson measures. Estimates are obtained for total variation and local metrics.…
Using techniques from Poisson approximation, we prove explicit error bounds on the number of permutations that avoid any pattern. Most generally, we bound the total variation distance between the joint distribution of pattern occurrences…
This paper deals with Poisson approximation to weighted sums of locally dependent random variables using Stein's method. The derived result represents a significant improvement of existing results. To illustrate the effectiveness of our…
The maximum entropy approach operating with quite general entropy measure and constraint is considered. It is demonstrated that for a conditional or parametrized probability distribution $f(x|\mu)$ there is a "universal" relation among the…
New upper bounds on the relative entropy are derived as a function of the total variation distance. One bound refines an inequality by Verd\'{u} for general probability measures. A second bound improves the tightness of an inequality by…
Here we present an analytic approximation for the entropy of floating-point numbers, along with bounds on the error of this approximation. It is well-known that the differential entropy is tightly linked to the discrete entropy of a…
In this paper, we develop a general theory on the coverage probability of random intervals defined in terms of discrete random variables with continuous parameter spaces. The theory shows that the minimum coverage probabilities of random…
We study translation-invariant determinantal random point fields on the real line. We prove, under quite general conditions, that the smallest nearest spacings between the particles in a large interval have Poisson statistics as the length…
We establish the first known upper bound on the exact and Wyner's common information of $n$ continuous random variables in terms of the dual total correlation between them (which is a generalization of mutual information). In particular, we…
Measuring dependence between random variables is a fundamental problem in Statistics, with applications across diverse fields. While classical measures such as Pearson's correlation have been widely used for over a century, they have…
The maximum-entropy remote sampling problem (MERSP) is to select a subset of s random variables from a set of n random variables, so as to maximize the information concerning a set of target random variables that are not directly…
A quasi two-dimensional colloidal suspension is studied under the influence of immobilisation (pinning) of a random fraction of its particles. We introduce a novel experimental method to perform random pinning and, with the support of…
Motivated by the approach of random linear codes, a new distance in the vector space over a finite field is defined as the logarithm of the "surface area" of a Hamming ball with radius being the corresponding Hamming distance. It is named…
For arbitrary two probability measures on real d-space with given means and variances (covariance matrices), we provide lower bounds for their total variation distance. In the one-dimensional case, a tight bound is given.
The likelihood function of a finite mixture model is a non-convex function with multiple local maxima and commonly used iterative algorithms such as EM will converge to different solutions depending on initial conditions. In this paper we…
We consider the problem of bounding large deviations for non-i.i.d. random variables that are allowed to have arbitrary dependencies. Previous works typically assumed a specific dependence structure, namely the existence of independent…
Rooted trees with probabilities are used to analyze properties of a variable length code. A bound is derived on the difference between the entropy rates of the code and a memoryless source. The bound is in terms of normalized informational…