Related papers: On the structure of multi-layer cellular neural ne…
Statistical learning in high-dimensional spaces is challenging without a strong underlying data structure. Recent advances with foundational models suggest that text and image data contain such hidden structures, which help mitigate the…
Drawing motivation from the manifold hypothesis, which posits that most high-dimensional data lies on or near low-dimensional manifolds, we apply manifold learning to the space of neural networks. We learn manifolds where datapoints are…
This paper explores the complexity of deep feedforward networks with linear pre-synaptic couplings and rectified linear activations. This is a contribution to the growing body of work contrasting the representational power of deep and…
Despite their increasing popularity and success in a variety of supervised learning problems, deep neural networks are extremely hard to interpret and debug: Given and already trained Deep Neural Net, and a set of test inputs, how can we…
The moduli space of the Calabi-Yau three-folds, which play a role as superstring ground states, exhibits the same {\em special geometry} that is known from nonlinear sigma models in $N=2$ supergravity theories. We discuss the symmetry…
We give new lower bounds for the (higher) topological complexity of a space, in terms of the Lusternik-Schnirelmann category of a certain auxiliary space. We also give new lower bounds for the rational topological complexity of a space, and…
Neural Collapse (NC) gives a precise description of the representations of classes in the final hidden layer of classification neural networks. This description provides insights into how these networks learn features and generalize well…
Recent evidence indicates that the abundance of recurring elementary interaction patterns in complex networks, often called subgraphs or motifs, carry significant information about their function and overall organization. Yet, the…
The success of deep neural networks in image classification and learning can be partly attributed to the features they extract from images. It is often speculated about the properties of a low-dimensional manifold that models extract and…
We will describe some results regarding the algorithmic nature of homeomorphism problems for manifolds; in particular, the following theorem. Theorem 1: Every PL or smooth simply connected manifold M^n of dimension n at least 5 can be…
Data analysis often concerns not only the space where data come from, but also various types of maps attached to data. In recent years, several related structures have been used to study maps on data, including Reeb spaces, mappers and…
Locally finite self-similar graphs with bounded geometry and without bounded geometry as well as non-locally finite self-similar graphs are characterized by the structure of their cell graphs. Geometric properties concerning the volume…
It is shown that the existence of a biseparating map between a large class of spaces of vector-valued continuous functions A(X,E) and A(Y,F) implies that some compactifications of X and Y are homeomorphic. In some cases, conditions are…
A technical note aiming to offer deeper intuition for the LayerNorm function common in deep neural networks. LayerNorm is defined relative to a distinguished 'neural' basis, but it does more than just normalize the corresponding vector…
Let $X$ be a topological space. A subset of $C(X)$, the space of continuous real-valued functions on $X$, is a partially ordered set in the pointwise order. Suppose that $X$ and $Y$ are topological spaces, and $A(X)$ and $A(Y)$ are subsets…
Given a factor code between sofic shifts X and Y, there is a family of decompositions of the original code into factor codes such that the entropies of the intermediate subshifts arising from the decompositions are dense in the interval…
It has been hypothesized that some form of "modular" structure in artificial neural networks should be useful for learning, compositionality, and generalization. However, defining and quantifying modularity remains an open problem. We cast…
In the context of classification problems, Deep Learning (DL) approaches represent state of art. Many DL approaches are based on variations of standard multi-layer feed-forward neural networks. These are also referred to as deep networks.…
The stunning empirical successes of neural networks currently lack rigorous theoretical explanation. What form would such an explanation take, in the face of existing complexity-theoretic lower bounds? A first step might be to show that…
There is a longstanding debate whether the Kolmogorov-Arnold representation theorem can explain the use of more than one hidden layer in neural networks. The Kolmogorov-Arnold representation decomposes a multivariate function into an…