Related papers: Mutual Dimension
We prove an exponential size separation between depth 2 and depth 3 neural networks (with real inputs), when approximating a $\mathcal{O}(1)$-Lipschitz target function to constant accuracy, with respect to a distribution with support in the…
We give explicit theoretical and heuristical bounds for how big does a data set sampled from a reach-1 submanifold M of euclidian space need to be, to be able to estimate the dimension of M with 90% confidence.
Quantifying degrees of fusion and separability between data groups in representation space is a fundamental problem in representation learning, particularly under domain shift. A meaningful metric should capture fusion-altering factors like…
We argue that the estimation of mutual information between high dimensional continuous random variables can be achieved by gradient descent over neural networks. We present a Mutual Information Neural Estimator (MINE) that is linearly…
A matrix $T \in \M_n(\C)$ is \emph{UECSM} if it is unitarily equivalent to a complex symmetric (i.e., self-transpose) matrix. We develop several techniques for studying this property in dimensions three and four. Among other things, we…
We show that, for disjoint domains in the Euclidean space whose boundaries satisfy a non-degeneracy condition, mutual absolute continuity of their harmonic measures implies absolute continuity with respect to surface measure and…
Let $\mathbf{Y}$ be the solution space of an $n$-layer cellular neural network, and let $\mathbf{Y}^{(i)}$ and $\mathbf{Y}^{(j)}$ be the hidden spaces, where $1 \leq i, j \leq n$. ($\mathbf{Y}^{(n)}$ is called the output space.) The…
We study approximation of embeddings between finite dimensional L_p spaces in the quantum model of computation. For the quantum query complexity of this problem matching (up to logarithmic factors) upper and lower bounds are obtained. The…
The conditional mutual information I(X;Y|Z) measures the average information that X and Y contain about each other given Z. This is an important primitive in many learning problems including conditional independence testing, graphical model…
We study the \emph{upper regularity dimension} which describes the extremal local scaling behaviour of a measure and effectively quantifies the notion of \emph{doubling}. We conduct a thorough study of the upper regularity dimension,…
We prove various results connected together by the common thread of computability theory. First, we investigate a new notion of algorithmic dimension, the inescapable dimension, which lies between the effective Hausdorff and packing…
Consider a set of points sampled independently near a smooth compact submanifold of Euclidean space. We provide mathematically rigorous bounds on the number of sample points required to estimate both the dimension and the tangent spaces of…
Rectangles are used to approximate objects, or sets of objects, in a plethora of applications, systems and index structures. Many tasks, such as nearest neighbor search and similarity ranking, require to decide if objects in one rectangle A…
We derive the calculation of two critical numbers predicting the behavior of perceptron networks. First, we derive the calculation of what we call the lossless memory (LM) dimension. The LM dimension is a generalization of the…
We introduce three measures of complexity for families of sets. Each of the three measures, that we call dimensions, is defined in terms of the minimal number of convex subfamilies that are needed for covering the given family: for upper…
Let $X = G/\Gamma$, where $G$ is a Lie group and $\Gamma$ is a lattice in $G$, and let $U$ be a subset of $X$ whose complement is compact. We use the exponential mixing results for diagonalizable flows on $X$ to give upper estimates for the…
The merit of projecting data onto linear subspaces is well known from, e.g., dimension reduction. One key aspect of subspace projections, the maximum preservation of variance (principal component analysis), has been thoroughly researched…
Pairwise Euclidean distance calculation is a fundamental step in many machine learning and data analysis algorithms. In real-world applications, however, these distances are frequently distorted by heteroskedastic noise$\unicode{x2014}$a…
Euclidean embeddings of data are fundamentally limited in their ability to capture latent semantic structures, which need not conform to Euclidean spatial assumptions. Here we consider an alternative, which embeds data as discrete…
We prove the Courtade-Kumar conjecture, which states that the mutual information between any Boolean function of an $n$-dimensional vector of independent and identically distributed inputs to a memoryless binary symmetric channel and the…