Related papers: Distributed Simulation of Continuous Random Variab…
This paper develops upper and lower bounds for the probability of Boolean functions by treating multiple occurrences of variables as independent and assigning them new individual probabilities. We call this approach dissociation and give an…
An information-theoretic development is given for the problem of compound Poisson approximation, which parallels earlier treatments for Gaussian and Poisson approximation. Let $P_{S_n}$ be the distribution of a sum $S_n=\Sumn Y_i$ of…
We define a measure of redundant information based on projections in the space of probability distributions. Redundant information between random variables is information that is shared between those variables. But in contrast to mutual…
The existing upper and lower bounds between entropy and error are mostly derived through an inequality means without linking to joint distributions. In fact, from either theoretical or application viewpoint, there exists a need to achieve a…
Bounds on information combining are entropic inequalities that determine how the information, or entropy, of a set of random variables can change when they are combined in certain prescribed ways. Such bounds play an important role in…
Many inference problems involving questions of optimality ask for the maximum or the minimum of a finite set of unknown quantities. This technical report derives the first two posterior moments of the maximum of two correlated Gaussian…
In this paper we generalize the notion of common information of two dependent variables introduced by G\'acs & K\"orner. They defined common information as the largest entropy rate of a common random variable two parties observing one of…
In this paper, we derive generic bounds on the maximum deviations in prediction errors for sequential prediction via an information-theoretic approach. The fundamental bounds are shown to depend only on the conditional entropy of the data…
In this paper, explicit error bounds are derived in the approximation of rank $k$ projections of certain $n$-dimensional random vectors by standard $k$-dimensional Gaussian random vectors. The bounds are given in terms of $k$, $n$, and a…
Benford's law states that for many random variables X > 0 its leading digit D = D(X) satisfies approximately the equation P(D = d) = log_{10}(1 + 1/d) for d = 1,2,...,9. This phenomenon follows from another, maybe more intuitive fact,…
Given a sample of independent and identically distributed random variables, a novel nonparametric maximum entropy method is presented to estimate the underlying continuous univariate probability density function (pdf). Estimates are found…
Coherent lower previsions are general probabilistic models allowing incompletely specified probability distributions. However, for complete description of a coherent lower prevision -- even on finite underlying sample spaces -- an infinite…
In this paper, we revisit the discrete lossy Gray-Wyner problem. In particular, we derive its optimal second-order coding rate region, its error exponent (reliability function) and its moderate deviations constant under mild conditions on…
We consider a standard two-source model for uniform common randomness (UCR) generation, in which Alice and Bob observe independent and identically distributed (i.i.d.) samples of a correlated finite source and where Alice is allowed to send…
Shearer's inequality bounds the sum of joint entropies of random variables in terms of the total joint entropy. We give another lower bound for the same sum in terms of the individual entropies when the variables are functions of…
The distribution of the sum of independent identically distributed uniform random variables is well-known. However, it is sometimes necessary to analyze data which have been drawn from different uniform distributions. By inverting the…
Differential entropy and log determinant of the covariance matrix of a multivariate Gaussian distribution have many applications in coding, communications, signal processing and statistical inference. In this paper we consider in the high…
In this paper, tight upper and lower bounds are derived on the weighted sum of minimum mean-squared errors for additive Gaussian noise channels. The bounds are obtained by constraining the input distribution to be close to a Gaussian…
This paper considers a variation of the full-information secretary problem where the random variables to be observed are independent but not necessary identically distributed. The main result is a sharp lower bound for the optimal win…
L\'evy's Upward Theorem says that the conditional expectation of an integrable random variable converges with probability one to its true value with increasing information. In this paper, we use methods from effective probability theory to…