Related papers: Compound Poisson Approximation via Information Fun…
The concept of Shannon entropy of random variables was generalized to measurable functions in general, and to simple functions with finite values in particular. It is shown that the information measure of a function is related to the time…
We present a general approach, based on exponential inequalities, to derive bounds on the generalization error of randomized learning algorithms. Using this approach, we provide bounds on the average generalization error as well as bounds…
A thorough account is given of the derivation of uniform semiclassical approximations to the particle and kinetic energy densities of N noninteracting bounded fermions in one dimension. The employed methodology allows the inclusion of…
Sufficient conditions are developed, under which the compound Poisson distribution has maximal entropy within a natural class of probability measures on the nonnegative integers. Recently, one of the authors [O. Johnson, {\em Stoch. Proc.…
This paper proposes a unifying variational approach for proving and extending some fundamental information theoretic inequalities. Fundamental information theory results such as maximization of differential entropy, minimization of Fisher…
Given a decision process based on the approximate probability density function returned by a data assimilation algorithm, an interaction level between the decision making level and the data assimilation level is designed to incorporate the…
Assume that we observe a sample of size n composed of p-dimensional signals, each signal having independent entries drawn from a scaled Poisson distribution with an unknown intensity. We are interested in estimating the sum of the n unknown…
Results of numerical procedure of constructing confidence intervals for parameter of the Poisson distribution of signal events in the presence of background events with known value of parameter of Poisson distribution are presented. It is…
Let $(Y_i,Z_i)_{i\geq 1}$ be a sequence of independent, identically distributed (i.i.d.) random vectors taking values in $\RRR^k\times\RRR^d$, for some integers $k$ and $d$. Given $z\in \RRR^d$, we provide a nonstandard functional limit law…
We combine the method of exchangeable pairs with Stein's method for functional approximation. As a result, we give a general linearity condition under which an abstract Gaussian approximation theorem for stochastic processes holds. We apply…
Consider an unlimited homogeneous medium disturbed by points generated via Poisson process. The neighborhood of a point plays an important role in spatial statistics problems. Here, we obtain analytically the distance statistics to $k$th…
We consider the estimation of a signal from the knowledge of its noisy linear random Gaussian projections. A few examples where this problem is relevant are compressed sensing, sparse superposition codes, and code division multiple access.…
In this paper we establish lower bounds on information divergence from a distribution to certain important classes of distributions as Gaussian, exponential, Gamma, Poisson, geometric, and binomial. These lower bounds are tight and for…
We consider an \eps-approximation by n-term partial sums of the Karhunen-Lo\`eve expansion to d-parametric random fields of tensor product-type in the average case setting. We investigate the behavior, as d tends to infinity, of the…
We derive general upper bounds to pointwise mutual information in terms of stochastic Fisher information and show these bounds average to known results in the literature for bounds to mutual information in terms of Fisher information. These…
We study the normal approximation of functionals of Poisson measures having the form of a finite sum of multiple integrals. When the integrands are nonnegative, our results yield necessary and sufficient conditions for central limit…
Markov Random Field models are powerful tools for the study of complex systems. However, little is known about how the interactions between the elements of such systems are encoded, especially from an information-theoretic perspective. In…
Let $Y=X_1+\cdots+X_N$ be a sum of a random number of exchangeable random variables, where the random variable $N$ is independent of the $X_j$, and the $X_j$ are from the generalized multinomial model introduced by Tallis (1962). This…
Information theory is built on probability measures and by definition a probability measure has total mass 1. Probability measures are used to model uncertainty, and one may ask how important it is that the total mass is one. We claim that…
The paper develops new methods of non-parametric estimation a compound Poisson distribution. Such a problem arise, in particular, in the inference of a Levy process recorded at equidistant time intervals. Our key estimator is based on…