Related papers: Thinning and Information Projections
Information divergence that measures the difference between two nonnegative matrices or tensors has found its use in a variety of machine learning problems. Examples are Nonnegative Matrix/Tensor Factorization, Stochastic Neighbor…
Given a correlation generated by a (possibly quantum) communication network, we study the amount of shared randomness required to generate it. We develop a novel upper bound for approximating distributions generated by arbitrary networks…
A class of subsets designated as very thin subsets of natural numbers has been studied and seen that theory of convergence may be rediscovered if very thin sets are given to play main role instead of thin or finite sets which removes some…
We obtain tight bounds on Poisson tails which are easy to handle. A short proof based on the median of the gamma distribution is given. Numerical comparisons with other known estimates are made. As an application, we consider the rates of…
The goal of this paper is to provide theorems on convergence rates of posterior distributions that can be applied to obtain good convergence rates in the context of density estimation as well as regression. We show how to choose priors so…
In this paper, we first establish general bounds on the Fisher information distance to the class of normal distributions of Malliavin differentiable random variables. We then study the rate of Fisher information convergence in the central…
The aim of the present work is to show that the results obtained earlier on the approximation of distributions of sums of independent terms by the accompanying compound Poisson laws may be interpreted as rather sharp quantitative estimates…
We unify f-divergences, Bregman divergences, surrogate loss bounds (regret bounds), proper scoring rules, matching losses, cost curves, ROC-curves and information. We do this by systematically studying integral and variational…
We present novel bounds for estimating discrete probability distributions under the $\ell_\infty$ norm. These are nearly optimal in various precise senses, including a kind of instance-optimality. Our data-dependent convergence guarantees…
An information-theoretic upper bound on the generalization error of supervised learning algorithms is derived. The bound is constructed in terms of the mutual information between each individual training sample and the output of the…
It is shown that the diffusion equation and its adjoint (time reversed) equation can be derived with only a few assumptions, using an information-theoretic approach based on the principle of minimum Fisher information
We prove a central limit theorem for a sequence of random variables whose means are ambiguous and vary in an unstructured way. Their joint distribution is described by a set of measures. The limit is (not the normal distribution and is)…
We consider a random interval splitting process, in which the splitting rule depends on the empirical distribution of interval lengths. We show that this empirical distribution converges to a limit almost surely as the number of intervals…
We derive an information-theoretic lower bound for sample complexity in sparse recovery problems where inputs can be chosen sequentially and adaptively. This lower bound is in terms of a simple mutual information expression and unifies many…
This work provides some general theorems about unconditional and conditional weak convergence of empirical processes in the case of Poisson sampling designs. The theorems presented in this work are stronger than previously published…
We consider a notion of uniform thinning for a finite sequence of random variables $(X_1,...,X_n)$ obtained by removing one random variable, uniformly at random. If a triangular array of random variables $(X_{n,k} : n \in \mathbb{N}_+, 1…
In this paper, tight upper and lower bounds are derived on the weighted sum of minimum mean-squared errors for additive Gaussian noise channels. The bounds are obtained by constraining the input distribution to be close to a Gaussian…
This paper is on developing some computer-assisted proof methods involving non-classical inequalities for Shannon entropy. Two areas of the applications of information inequalities are studied: Secret sharing schemes and hat guessing games.…
Denoising diffusion models have spurred significant gains in density modeling and image generation, precipitating an industrial revolution in text-guided AI art generation. We introduce a new mathematical foundation for diffusion models…
We investigate the rate of convergence in the central limit theorem for convex sets. We obtain bounds with a power-law dependence on the dimension. These bounds are asymptotically better than the logarithmic estimates which follow from the…