Related papers: Entropy Bounds for Discrete Random Variables via M…
We seek an entropy estimator for discrete distributions with fully empirical accuracy bounds. As stated, this goal is infeasible without some prior assumptions on the distribution. We discover that a certain information moment assumption…
We prove a Bernstein-type bound for the difference between the average of negative log-likelihoods of independent discrete random variables and the Shannon entropy, both defined on a countably infinite alphabet. The result holds for the…
We consider random i.i.d. samples of absolutely continuous measures on bounded connected domains. We prove an upper bound on the $\infty$-transportation distance between the measure and the empirical measure of the sample. The bound is…
This paper develops upper and lower bounds for the probability of Boolean functions by treating multiple occurrences of variables as independent and assigning them new individual probabilities. We call this approach dissociation and give an…
We establish a sharp estimate for a minimal number of binary digits (bits) needed to represent all bounded total generalized variation functions taking values in a general totally bounded metric space $(E,\rho)$ up to an accuracy of…
Making statistical predictions requires tackling two problems: one must assign appropriate probability distributions and then one must calculate a variety of expected values. The method of maximum entropy is commonly used to address the…
Within the task of collaborative filtering two challenges for computing conditional probabilities exist. First, the amount of training data available is typically sparse with respect to the size of the domain. Thus, support for higher-order…
The question of optimally approximating an arbitrary probability measure in the Wasserstein distance by a discrete one with uniform weights is considered. Estimates are obtained for the optimal approximation distance, with an explicit rate…
This work introduces a new, explicit bound on the Hellinger distance between a continuous random variable and a Gaussian with matching mean and variance. As example applications, we derive a quantitative Hellinger central limit theorem and…
The mutual information (MI) between two random variables is an important correlation measure in data analysis. The Shannon entropy of a joint probability distribution is the variable part under fixed marginals. We aim to minimize and…
Defining a divergence between the laws of continuous martingales is a delicate task, owing to the fact that these laws tend to be singular to each other. An important idea, put forward by N. Gantert, is to instead consider a scaling limit…
We construct a new sufficient conditions for boundedness or continuity of arbitrary random fields relying on the so-called partition scheme, alike in the classical majorizing measure method. We deduce also the used in the practice…
It is well known that a binomial $(n,p)$ can be approximated by a Poisson distribution with parameter $np$. The typical approach in undergraduate probability texts is to show a convergence result for the distribution of the binomial as $n$…
We introduce an independence criterion based on entropy regularized optimal transport. Our criterion can be used to test for independence between two samples. We establish non-asymptotic bounds for our test statistic and study its…
We study the asymptotic behavior of the maximum interpoint distance of random points in a $d$-dimensional set with a unique diameter and a smooth boundary at the poles. Instead of investigating only a fixed number of $n$ points as $n$ tends…
We introduce variants of relative entropy of entanglement based on the optimal distinguishability from unentangled states by means of restricted measurements. In this way, we are able to prove that the standard regularized entropy of…
We present a framework for obtaining explicit bounds on the rate of convergence to equilibrium of a Markov chain on a general state space, with respect to both total variation and Wasserstein distances. For Wasserstein bounds, our main tool…
We investigate the R\'enyi entropy of independent sums of integer valued random variables through Fourier theoretic means, and give sharp comparisons between the variance and the R\'enyi entropy, for Poisson-Bernoulli variables. As…
The maximum entropy principle is a powerful tool for solving underdetermined inverse problems. This paper considers the problem of discretizing a continuous distribution, which arises in various applied fields. We obtain the approximating…
In this paper, we obtain error bound for binomial and negative binomial approximations to weighted sums of locally dependent random variables, using Stein's method. We also discuss approximation results for weighted sums of independent…