Related papers: Thinning and Information Projections
Current discrete randomness and information conservation inequalities are over total recursive functions, i.e. restricted to deterministic processing. This restriction implies that an algorithm can break algorithmic randomness conservation…
We present a new proof rule for verifying lower bounds on quantities of probabilistic programs. Our proof rule is not confined to almost-surely terminating programs -- as is the case for existing rules -- and can be used to establish…
In this paper we develop tools for studying limit theorems by means of convexity. We establish bounds for the discrepancy in total variation between probability measures $\mu$ and $\nu$ such that $\nu$ is log-concave with respect to $\mu$.…
This paper proves several weak limit theorems for the joint version of extreme order statistics and partial sums of independently and identically distributed random variables. The results are also extended to almost sure limit version.
We derive tight and computable bounds on the bias of statistical estimators, or more generally of quantities of interest, when evaluated on a baseline model P rather than on the typically unknown true model Q. Our proposed method combines…
We consider the Gaussian limit law for the distribution of the middle prime factor of an integer, defined according to multiplicity or not. We obtain an optimal bound for the speed of convergence, thereby improving on previous estimates…
Information projections are the key building block of variational inference algorithms and are used to approximate a target probabilistic model by projecting it onto a family of tractable distributions. In general, there is no guarantee on…
Consider an infinite sequence of independent, uniformly chosen points from $[0,1]^d$. After looking at each point in the sequence, an overseer is allowed to either keep it or reject it, and this choice may depend on the locations of all…
Observations on the past provide some hints about what will happen in the future, and this can be quantified using information theory. The ``predictive information'' defined in this way has connections to measures of complexity that have…
The Shannon lower bound is one of the few lower bounds on the rate-distortion function that holds for a large class of sources. In this paper, it is demonstrated that its gap to the rate-distortion function vanishes as the allowed…
We derive a tight lower bound on equivocation (conditional entropy), or equivalently a tight upper bound on mutual information between a signal variable and channel outputs. The bound is in terms of the joint distribution of the signals and…
The binary divergences that are divergences between probability measures defined on the same 2-point set have an interesting property. For the chi-squared divergence and the relative entropy, it is known that their binary divergence attain…
We develop, discuss, and compare several inference techniques to constrain theory parameters in collider experiments. By harnessing the latent-space structure of particle physics processes, we extract extra information from the simulator.…
We consider the problem of parameter estimation in a Bayesian setting and propose a general lower-bound that includes part of the family of $f$-Divergences. The results are then applied to specific settings of interest and compared to other…
Finding the underlying probability distributions of a set of observed sequences under the constraint that each sequence is generated i.i.d by a distinct distribution is considered. The number of distributions, and hence the number of…
We derive conditions under which random sequences of polarizations (two-point symmetrizations) converge almost surely to the symmetric decreasing rearrangement. The parameters for the polarizations are independent random variables whose…
In this paper we introduce a notion of tightness for a family of nonlinear expectations and show that the tightness can be applied to obtain weak compactness in a framework of nonlinear expectation space. This criterion is very useful for…
The central limit theorem is one of the most fundamental results in probability and has been successfully extended to locally dependent data and strongly-mixing random fields. In this paper, we establish its rate of convergence for…
This paper studies probabilistic rates of convergence for consensus+innovations type of algorithms in random, generic networks. For each node, we find a lower and also a family of upper bounds on the large deviations rate function, thus…
In applied probability, the normal approximation is often used for the distribution of data with assumed additive structure. This tradition is based on the central limit theorem for sums of (independent) random variables. However, it is…