Related papers: Bounds on the Information Divergence for Hypergeom…
An information-theoretic development is given for the problem of compound Poisson approximation, which parallels earlier treatments for Gaussian and Poisson approximation. Let $P_{S_n}$ be the distribution of a sum $S_n=\Sumn Y_i$ of…
We introduce and discuss the concept of Gaussian probability density function (pdf) for the n-dimensional hyperbolic space which has been proposed as an environment for coding and decoding signals. An upper bound for the error probability…
The Fisher information matrix (FIM) plays an important role in the analysis of parameter inference and system design problems. In a number of cases, however, the statistical data distribution and its associated information matrix are either…
In this note we discuss additional properties of mixed Poisson distributions. We discuss the convergence of mixed Poisson distributions to its mixing distribution for the scaling parameter tending to infinity. Moreover, we obtain a central…
Recent advances in statistical learning theory have revealed profound connections between mutual information (MI) bounds, PAC-Bayesian theory, and Bayesian nonparametrics. This work introduces a novel mutual information bound for…
The belief propagation (BP) algorithm is widely applied to perform approximate inference on arbitrary graphical models, in part due to its excellent empirical properties and performance. However, little is known theoretically about when…
This is an expository article on the Poisson binomial distribution. We review lesser known results and recent progress on this topic, including geometry of polynomials and distribution learning. We also provide examples to illustrate the…
We explore asymptotically optimal bounds for deviations of Bernoulli convolutions from the Poisson limit in terms of the Shannon relative entropy and the Pearson $\chi^2$-distance. The results are based on proper non-uniform estimates for…
This paper is about Information Geometry, a relatively new subject within mathematical statistics that attempts to study the problem of inference by using tools from modern differential geometry. This paper provides an overview of some of…
We present a new method to propagate lower bounds on conditional probability distributions in conventional Bayesian networks. Our method guarantees to provide outer approximations of the exact lower bounds. A key advantage is that we can…
While Shannon's mutual information has widespread applications in many disciplines, for practical applications it is often difficult to calculate its value accurately for high-dimensional variables because of the curse of dimensionality.…
There is currently growing interest in modeling the information diffusion on social networks across multi-disciplines. The majority of the corresponding research has focused on information diffusion independently, ignoring the network…
In transfer learning, training and testing data sets are drawn from different data distributions. The transfer generalization gap is the difference between the population loss on the target data distribution and the training loss. The…
The multivariate hypergeometric distribution describes sampling without replacement from a discrete population of elements divided into multiple categories. Addressing a gap in the literature, we tackle the challenge of estimating discrete…
We consider a hypothesis testing problem where a part of data cannot be observed. Our helper observes the missed data and can send us a limited amount of information about them. What kind of this limited information will allow us to make…
Many problems of interest in computer science and information theory can be phrased in terms of a probability distribution over discrete variables associated to the vertices of a large (but finite) sparse graph. In recent years,…
Tight bounds for several symmetric divergence measures are introduced, given in terms of the total variation distance. Each of these bounds is attained by a pair of 2 or 3-element probability distributions. An application of these bounds…
We examine a generalization of the binomial distribution associated with a strictly increasing sequence of numbers and we prove its Poisson-like limit. Such generalizations might be found in quantum optics with imperfect detection. We…
Recently, contagion-based (disease, information, etc.) spreading on social networks has been extensively studied. In this paper, other than traditional full interaction, we propose a partial interaction based spreading model, considering…
This paper focuses on parameter estimation and introduces a new method for lower bounding the Bayesian risk. The method allows for the use of virtually \emph{any} information measure, including R\'enyi's $\alpha$, $\varphi$-Divergences, and…