Related papers: Mutual information matrices are not always positiv…

Diffusion bridge models have recently become a powerful tool in the field of generative modeling. In this work, we leverage their power to address another important problem in machine learning and information theory, the estimation of the…

We examine a class of deep learning models with a tractable method to compute information-theoretic quantities. Our contributions are three-fold: (i) We show how entropies and mutual informations can be derived from heuristic statistical…

Positivity, the assumption that every unique combination of confounding variables that occurs in a population has a non-zero probability of an action, can be further delineated as deterministic positivity and stochastic positivity. Here, we…

Estimating mutual information (MI) between two continuous random variables $X$ and $Y$ allows to capture non-linear dependencies between them, non-parametrically. As such, MI estimation lies at the core of many data science applications.…

We studied the mutual information between a stimulus and a large system consisting of stochastic, statistically independent elements that respond to a stimulus. The Mutual Information (MI) of the system saturates exponentially with system…

Conditional Mutual Information (CMI) is a measure of conditional dependence between random variables X and Y, given another random variable Z. It can be used to quantify conditional dependence among variables in many data-driven inference…

Let X1, ..., Xn be arbitrary non-negative independent random variables with respective expected values $\mu_{i}$ at most one. We sketch but do not prove an equivalent conjecture to Feige's Conjecture $\mathbb{P} \left( \sum_{i=1}^{n} X_{i}…

This correspondence studies the basic problem of classifications - how to evaluate different classifiers. Although the conventional performance indexes, such as accuracy, are commonly used in classifier selection or evaluation,…

This paper studies a high-dimensional inference problem involving the matrix tensor product of random matrices. This problem generalizes a number of contemporary data science problems including the spiked matrix models used in sparse…

A complex Hilbert space of dimension six supports at least three but not more than seven mutually unbiased bases. Two computer-aided analytical methods to tighten these bounds are reviewed, based on a discretization of parameter space and…

In this paper, we investigate the invertibility of sparse symmetric matrices. We show that for an $n\times n$ sparse symmetric random matrix $A$ with $A_{ij} = \delta_{ij} \xi_{ij}$ is invertible with high probability. Here, $\delta_{ij}$s,…

We establish a lower bound on the entropy of weighted sums of (possibly dependent) random variables $(X_1, X_2, \dots, X_n)$ possessing a symmetric joint distribution. Our lower bound is in terms of the joint entropy of $(X_1, X_2, \dots,…

Mutual information (MI) is one of the most general ways to measure relationships between random variables, but estimating this quantity for complex systems is challenging. Denoising diffusion models have recently set a new bar for density…

We show that any symmetric positive definite homogeneous matrix polynomial $M\in\R[x_1,...,x_n]^{m\times m}$ admits a piecewise semi-certificate, i.e. a collection of identites $M(x)=\sum_jf_{i,j}(x)U_{i,j}(x)^TU_{i,j}(x)$ where…

Let K, K' be convex cones residing in finite-dimensional real vector spaces E, E'. An element in the tensor product E \otimes E' is K \otimes K'-separable if it can be represented as finite sum \sum_l x_l \otimes x'_l with x_l \in K and…

We study the problem of community detection when there is covariate information about the node labels and one observes multiple correlated networks. We provide an asymptotic upper bound on the per-node mutual information as well as a…

This survey contains a selection of topics unified by the concept of positive semi-definiteness (of matrices or kernels), reflecting natural constraints imposed on discrete data (graphs or networks) or continuous objects (probability or…

An elliptic random matrix $X$ is a square matrix whose $(i,j)$-entry $X_{ij}$ is independent of the rest of the entries except possibly $X_{ji}$. Elliptic random matrices generalize Wigner matrices and non-Hermitian random matrices with…

The mutual information method has demonstrated to be very useful for deriving the potential order parameter of a system. Although the method suggests some constraints which help to define this quantity, there is still some freedom in the…

Mutual information has been successfully adopted in filter feature-selection methods to assess both the relevancy of a subset of features in predicting the target variable and the redundancy with respect to other variables. However,…