Related papers: Any-Dimensional Invariant Universality
A method for extracting multiscale geometric features from a data cloud is proposed and analyzed. The basic idea is to map each pair of data points into a real-valued feature function defined on $[0,1]$. The construction of these feature…
The introduction of convolutional layers greatly advanced the performance of neural networks on image tasks due to innately capturing a way of encoding and learning translation-invariant operations, matching one of the underlying symmetries…
Equivariant neural networks provide a principled framework for incorporating symmetry into learning architectures and have been extensively analyzed through the lens of their separation power, that is, the ability to distinguish inputs…
Let $\mathfrak{M}$ be a class of metric spaces. A metric space $Y$ is minimal $\mathfrak{M}$-universal if every $X\in\mathfrak{M}$ can be isometrically embedded in $Y$ but there are no proper subsets of $Y$ satisfying this property. We find…
The paper puts into discussion the concept of universality, in particular for structures not of the power of Turing computability. The question arises if for such structures a universal structure of the same kind exists or not. For that the…
Mathematical descriptions of dynamical systems are deeply rooted in topological spaces defined by non-Euclidean geometry. This paper proposes leveraging structure-rich geometric spaces for machine learning to achieve structural…
Symmetry groups allow to transform solutions of differential equations continuously into other solutions. This property can be used for the observability analysis of infinite-dimensional systems with input and output. In this contribution,…
The concept of dimension is essential to grasp the complexity of data. A naive approach to determine the dimension of a dataset is based on the number of attributes. More sophisticated methods derive a notion of intrinsic dimension (ID)…
In many practical applications it is important to build symmetries into neural network architectures. Consider the important case of permutation symmetry on point clouds consisting of $n$ points in $d$ dimensions. In this case the network…
A main object of our study is multiset functions -- that is, permutation-invariant functions over inputs of varying sizes. Deep Sets, proposed by \cite{zaheer2017deep}, provides a \emph{universal representation} for continuous multiset…
The paper proposes a vector generalization of the basic concepts of the theory of complex variable: the concept of modulus and argument of complex number. The author introduces some generalizations of the notion of holomorphic functions and…
Deep Neural Networks (DNNs) are universal function approximators providing state-of- the-art solutions on wide range of applications. Common perceptual tasks such as speech recognition, image classification, and object tracking are now…
Recent years have been marked with the fast-pace diversification and increasing ubiquity of machine learning applications. Yet, a firm theoretical understanding of the surprising efficiency of neural networks to learn from high-dimensional…
High-dimensional big data appears in many research fields such as image recognition, biology and collaborative filtering. Often, the exploration of such data by classic algorithms is encountered with difficulties due to `curse of…
We prove that every nonnegative continuous real-valued function on a given compact metric space is the uniform limit of some increasing sequence of nonnegative simple functions being linear combinations of indicators of open sets; here the…
Recently, it was realized that anomalies can be completely classified by topological orders, symmetry protected topological (SPT) orders, and symmetry enriched topological orders in one higher dimension. The anomalies that people used to…
It is pointed out that the universality might seriously be violated by models with several fixed points.
How sensitive should machine learning models be to input changes? We tackle the question of model smoothness and show that it is a useful inductive bias which aids generalization, adversarial robustness, generative modeling and…
Deep neural networks are widely used in various domains. However, the nature of computations at each layer of the deep networks is far from being well understood. Increasing the interpretability of deep neural networks is thus important.…
A key problem in approximation theory is the recovery of high-dimensional functions from samples. In many cases, the functions of interest exhibit anisotropic smoothness, and, in many practical settings, the nature of this anisotropy may be…