Related papers: Bounds on the Information Divergence for Hypergeom…
Convergence bounds are one of the main tools to obtain information on the performance of a distributed machine learning task, before running the task itself. In this work, we perform a set of experiments to assess to which extent, and in…
The binomial and Poisson distributions have interesting relationships with the beta and gamma distributions, respectively, which involve their cumulative distribution functions and the use of conjugate priors in Bayesian statistics. We…
In this paper a numerical method is presented, which finds a lower bound for the mutual information between a binary and an arbitrary finite random variable with joint distributions that have a variational distance not greater than a known…
The following problem is considered: given a joint distribution $P_{XY}$ and an event $E$, bound $P_{XY}(E)$ in terms of $P_XP_Y(E)$ (where $P_XP_Y$ is the product of the marginals of $P_{XY}$) and a measure of dependence of $X$ and $Y$.…
The statistics and machine learning communities have recently seen a growing interest in classification-based approaches to two-sample testing. The outcome of a classification-based two-sample test remains a rejection decision, which is not…
The problem of overdispersion in multivariate count data is a challenging issue. Nowadays, it covers a central role mainly due to the relevance of modern technologies data, such as Next Generation Sequencing and textual data from the web or…
A novel, non-trivial, probabilistic upper bound on the entropy of an unknown one-dimensional distribution, given the support of the distribution and a sample from that distribution, is presented. No knowledge beyond the support of the…
Gaussian belief propagation (BP) has been widely used for distributed estimation in large-scale networks such as the smart grid, communication networks, and social networks, where local measurements/observations are scattered over a wide…
Statistical divergences are important tools in data analysis, information theory, and statistical physics, and there exist well known inequalities on their bounds. However, in many circumstances involving temporal evolution, one needs…
Mixture distributions are extensively used as a modeling tool in diverse areas from machine learning to communications engineering to physics, and obtaining bounds on the entropy of probability distributions is of fundamental importance in…
Diffusion models are one of the most important families of deep generative models. In this note, we derive a quantitative upper bound on the Wasserstein distance between the data-generating distribution and the distribution learned by a…
Estimation of the $\phi$-divergence between two unknown probability distributions using empirical data is a fundamental problem in information theory and statistical learning. We consider a multi-variate generalization of the data dependent…
Statistical inference more often than not involves models which are non-linear in the parameters thus leading to non-Gaussian posteriors. Many computational and analytical tools exist that can deal with non-Gaussian distributions, and…
This paper presents a stochastic geometry model for the investigation of fundamental information theoretic limitations in wireless networks. We derive a new unified multi-parameter cut-set bound on the capacity of networks of arbitrary…
This paper is on developing some computer-assisted proof methods involving non-classical inequalities for Shannon entropy. Two areas of the applications of information inequalities are studied: Secret sharing schemes and hat guessing games.…
We derive bounds for the Orlicz norm of the deviation of a random variable defined on $\mathbb{R}^n$ from its Gaussian mean value. The random variables are assumed to be smooth and the bound itself depends on the Orlicz norm of the…
Binomial data with unknown sizes often appear in biological and medical sciences and are usually overdispersed. All previous methods used parametric models and only considered overdispersion due to the variation of sizes. The proposed…
In this article, we obtain, for the total variance distance, the error bounds between Poisson and convolution of power series distributions via Stein's method. This provides a unified approach to many known discrete distributions. Several…
We examine the relationship between the mutual information between the output model and the empirical sample and the generalization of the algorithm in the context of stochastic convex optimization. Despite increasing interest in…
In this work, the probability of an event under some joint distribution is bounded by measuring it with the product of the marginals instead (which is typically easier to analyze) together with a measure of the dependence between the two…