Related papers: Mutual Dimension and Random Sequences
A well-known metric for quantifying the similarity between two clusterings is the adjusted mutual information. Compared to mutual information, a corrective term based on random permutations of the labels is introduced, preventing two…
Mixtures of high dimensional Gaussian distributions have been studied extensively in statistics and learning theory. While the total variation distance appears naturally in the sample complexity of distribution learning, it is analytically…
In this paper, we exhibit a new family of martingale couplings between two one-dimensional probability measures $\mu$ and $\nu$ in the convex order. This family is parametrised by two dimensional probability measures on the unit square with…
This report presents a new, algorithmic approach to the distributions of the distance between two points distributed uniformly at random in various polygons, based on the extended Kinematic Measure (KM) from integral geometry. We first…
Normalized mutual information is widely used as a similarity measure for evaluating the performance of clustering and classification algorithms. In this paper, we argue that results returned by the normalized mutual information are biased…
A constructive version of Hausdorff dimension is developed using constructive supergales, which are betting strategies that generalize the constructive supermartingales used in the theory of individual random sequences. This constructive…
We introduce notions of dimension of an infinite group, or more generally, a metric space, defined using percolation. Roughly speaking, the percolation dimension $pdim(G)$ of a group $G$ is the fastest rate of decay of a symmetric…
In this note we define and study graph invariants generalizing to higher dimension the maximum degree of a vertex and the vertex-connectivity (our $0$-dimensional cases). These are known to coincide almost surely in any regime for…
Randomly connected neural networks have long served as a theoretical tool for studying collective dynamics in neural populations, yet quantitative comparisons to experiments remain limited. Recent technological advances have made it…
Sliced mutual information (SMI) is defined as an average of mutual information (MI) terms between one-dimensional random projections of the random variables. It serves as a surrogate measure of dependence to classic MI that preserves many…
We study the mutual information estimation for mixed-pair random variables. One random variable is discrete and the other one is continuous. We develop a kernel method to estimate the mutual information between the two random variables. The…
Given a metric space with a Borel probability measure, for each integer $N$ we obtain a probability distribution on $N\times N$ distance matrices by considering the distances between pairs of points in a sample consisting of $N$ points…
There are several ways to measure the compressibility of a random measure; they include general approaches such as using the rate-distortion curve, as well as more specific notions, such as the Renyi information dimension (RID). The RID…
We define a measure of redundant information based on projections in the space of probability distributions. Redundant information between random variables is information that is shared between those variables. But in contrast to mutual…
The density matrix yields probabilistic information about the outcome of measurements on a quantum system, but it does not distinguish between classical randomness in the preparation of the system and entanglement with its environment.…
Mutual information is an important measure of the dependence among variables. It has become widely used in statistics, machine learning, biology, etc. However, the standard techniques for estimating it often perform poorly in higher…
Mutual information (MI) is a fundamental measure of statistical dependence between two variables, yet accurate estimation from finite data remains notoriously difficult. No estimator is universally reliable, and common approaches fail in…
Probabilistic frames are a generalization of finite frames into the Wasserstein space of probability measures with finite second moment. We introduce new probabilistic definitions of duality, analysis, and synthesis and investigate their…
Randomness (in the sense of being generated in an IID fashion) and exchangeability are standard assumptions in nonparametric statistics and machine learning, and relations between them have been a popular topic of research. This short paper…
Comparing probability distributions is a core challenge across the natural, social, and computational sciences. Existing methods, such as Maximum Mean Discrepancy (MMD), struggle in high-dimensional and non-compact domains. Here we…