Related papers: Mutual Dimension
The mutual information characterizes correlations between spatially separated regions of a system. Yet, in experiments we often measure dynamical correlations, which involve probing operators that are also separated in time. Here, we…
Leveraging the intrinsic symmetries in data for clear and efficient analysis is an important theme in signal processing and other data-driven sciences. A basic example of this is the ubiquity of the discrete Fourier transform which arises…
High dimensional data can have a surprising property: pairs of data points may be easily separated from each other, or even from arbitrary subsets, with high probability using just simple linear classifiers. However, this is more of a rule…
The study of inter-human communication requires a more complex framework than Shannon's (1948) mathematical theory of communication because "information" is defined in the latter case as meaningless uncertainty. Assuming that meaning cannot…
We formulate the conditional Kolmogorov complexity of x given y at precision r, where x and y are points in Euclidean spaces and r is a natural number. We demonstrate the utility of this notion in two ways. 1. We prove a point-to-set…
The cognitive framework of conceptual spaces [3] provides geometric means for representing knowledge. A conceptual space is a high-dimensional space whose dimensions are partitioned into so-called domains. Within each domain, the Euclidean…
Estimating mutual information (MI) is a fundamental yet challenging task in data science and machine learning. This work proposes a new estimator for mutual information. Our main discovery is that a preliminary estimate of the data…
Chemical space which encompasses all stable compounds is unfathomably large and its dimension scales linearly with the number of atoms considered. The success of machine learning methods suggests that many physical quantities exhibit…
We propose an axiomatic approach to the concept of an intrinsic dimension of a dataset, based on a viewpoint of geometry of high-dimensional structures. Our first axiom postulates that high values of dimension be indicative of the presence…
A mapping $f:X\to Y$ between metric spaces is called \emph{little Lipschitz} if the quantity $$ \operatorname{lip}(f(x)=\liminf_{r\to0}\frac{\operatorname{diam} f(B(x,r))}{r} $$ is finite for every $x\in X$. We prove that if a compact (or,…
We give two low-complexity algorithms, one for dimensionality reduction and one for dimensionality increase, which are applicable to any dataset, regardless of whether the set has an intrinsic dimension or not. The corresponding methods…
What is the dimension of a network? Here, we view it as the smallest dimension of Euclidean space into which nodes can be embedded so that pairwise distances accurately reflect the connectivity structure. We show that a recently proposed…
Maximal sets of mutually unbiased bases are useful throughout quantum physics, both in a foundational context and for applications. To date, it remains unknown if complete sets of mutually unbiased bases exist in Hilbert spaces of…
Algorithmic fractal dimensions quantify the algorithmic information density of individual points and may be defined in terms of Kolmogorov complexity. This work uses these dimensions to bound the classical Hausdorff and packing dimensions…
Let (X,Z) be a dynamical system on a compact metric X and let X be the countable union of closed invariant subsets X_i, i in N. We prove that mdim X =sup {mdim X_i : i in N}.
Consider an algorithm computing in a differential field with several commuting derivations such that the only operations it performs with the elements of the field are arithmetic operations, differentiation, and zero testing. We show that,…
We introduce the Mutual Information Machine (MIM), a novel formulation of representation learning, using a joint distribution over the observations and latent state in an encoder/decoder framework. Our key principles are symmetry and mutual…
Multidimensional scaling (MDS) is the act of embedding proximity information about a set of $n$ objects in $d$-dimensional Euclidean space. As originally conceived by the psychometric community, MDS was concerned with embedding a fixed set…
The information theoretic quantity known as mutual information finds wide use in classification and community detection analyses to compare two classifications of the same set of objects into groups. In the context of classification…
We characterize the algorithmic dimensions (i.e., the lower and upper asymptotic densities of information) of infinite binary sequences in terms of the inability of learning functions having an algorithmic constraint to detect patterns in…