Related papers: Bounds on the Information Divergence for Hypergeom…
We consider parameter estimation in distributed networks, where each sensor in the network observes an independent sample from an underlying distribution and has $k$ bits to communicate its sample to a centralized processor which computes…
A random geometric graph $G(\mathcal{X}_n, r_n)$ is formed by taking a binomial process $\mathcal{X}_n$ as the set of vertices and joining any two distinct points with an edge if they lie within distance $r_n$ of each other. We investigate…
Epidemic modeling has been extensively used in the last years in the field of telecommunications and computer networks. We consider the popular Susceptible-Infected-Susceptible spreading model as the metric for information spreading. In…
Hypothesis testing in high dimensional data is a notoriously difficult problem without direct access to competing models' likelihood functions. This paper argues that statistical divergences can be used to quantify the difference between…
Consider the family of power divergence statistics based on $n$ trials, each leading to one of $r$ possible outcomes. This includes the log-likelihood ratio and Pearson's statistic as important special cases. It is known that in certain…
We consider the problem of learning high-dimensional, nonparametric and structured (e.g. Gaussian) distributions in distributed networks, where each node in the network observes an independent sample from the underlying distribution and can…
Estimating mutual information (MI) is a fundamental yet challenging task in data science and machine learning. This work proposes a new estimator for mutual information. Our main discovery is that a preliminary estimate of the data…
This paper studies the propagation connectivity of a random hypergraph $\mathbb{G}$ containing both 2-edges and 3-hyperedges. We find an exact threshold of the propagation connectivity of $\mathbb{G}$: If $I_{\epsilon,r}<-1$, then…
This paper proposes an easy-to-compute upper bound for the overlap index between two probability distributions without requiring any knowledge of the distribution models. The computation of our bound is time-efficient and memory-efficient…
We correct claims about lower bounds on mutual information (MI) between real-valued random variables made in A. Kraskov {\it et al.}, Phys. Rev. E {\bf 69}, 066138 (2004). We show that non-trivial lower bounds on MI in terms of linear…
Domain divergence plays a significant role in estimating the performance of a model in new domains. While there is a significant literature on divergence measures, researchers find it hard to choose an appropriate divergence for a given NLP…
This paper is a strongly geometrical approach to the Fisher distance, which is a measure of dissimilarity between two probability distribution functions. The Fisher distance, as well as other divergence measures, are also used in many…
A pair of probability distributions over $\{0,1\}^n$ is said to be $(k,\delta)$-wise indistinguishable if all of the size $k$ marginals are within statistical distance at most $\delta$. Previous works introduced this concept and study when…
Randomness in scientific estimation is generally assumed to arise from unmeasured or uncontrolled factors. However, when combining subjective probability estimates, heterogeneity stemming from people's cognitive or information diversity is…
We derive distributional limits for empirical transport distances between probability measures supported on countable sets. Our approach is based on sensitivity analysis of optimal values of infinite dimensional mathematical programs and a…
Parametric inference posits a statistical model that is a specified family of probability distributions. Restricted inference, e.g., restricted likelihood ratio testing, attempts to exploit the structure of a statistical submodel that is a…
An identity between two versions of the Chernoff bound on the probability a certain large deviations event, is established. This identity has an interpretation in statistical physics, namely, an isothermal equilibrium of a composite system…
We use phase space distributions specifically, the Wigner distribution (WD) and Husimi distribution (HD) to investigate certain information-theoretic measures as descriptors for a given system. We extensively investigate and analyze…
Information divergence functions play a critical role in statistics and information theory. In this paper we show that a non-parametric f-divergence measure can be used to provide improved bounds on the minimum binary classification…
While efficient distribution learning is no doubt behind the groundbreaking success of diffusion modeling, its theoretical guarantees are quite limited. In this paper, we provide the first rigorous analysis on approximation and…