Related papers: A Survey on Universal Approximation Theorems

A challenging problem in many modern machine learning tasks is to process weight-space features, i.e., to transform or extract information from the weights and gradients of a neural network. Recent works have developed promising…

In recent years, a large class of nuclear $C^\ast$-algebras have been classified, modulo an assumption on the Universal Coefficient Theorem (UCT). We think this assumption is redundant and propose a strategy for proving it. Indeed,…

We prove simultaneous Universal Approximation of a certain type of Pade Approximants and of Taylor series with the same indexes. This is a generic phenomenon in the space of holomorphic functions in any simply connected domain, as well as…

While deep learning is successful in a number of applications, it is not yet well understood theoretically. A satisfactory theoretical characterization of deep learning however, is beginning to emerge. It covers the following questions: 1)…

We introduce an asymmetric operator of generalised translation, define the generalised modulus of smoothness by its means, and obtain the direct and inverse theorems in approximation theory for it.

\citet{farrell2021deep} establish non-asymptotic high-probability bounds for general deep feedforward neural network (with rectified linear unit activation function) estimators, with \citet[Theorem 1]{farrell2021deep} achieving a suboptimal…

Theory of representations of universal algebra is a natural development of the theory of universal algebra. In the book, I considered representation of universal algebra, diagram of representations and examples of representation. Morphism…

Artificial neural networks are simple and efficient machine learning tools. Defined originally in the traditional setting of simple vector data, neural network models have evolved to address more and more difficulties of complex real world…

We establish universal approximation theorems for infinite-dimensional geometric rough paths, i.e., we show that continuous functions on the space of infinite-dimensional weakly geometric H\"older continuous rough paths can be approximated…

We introduce compositional tensor trains (CTTs) for the approximation of multivariate functions, a class of models obtained by composing low-rank functions in the tensor-train format. This format can encode standard approximation tools,…

We develop a theory of category-equivariant neural networks (CENNs) that unifies group/groupoid-equivariant networks, poset/lattice-equivariant networks, graph and sheaf neural networks. Equivariance is formulated as naturality in a…

Finite Unified Theories (FUTs) are N=1 supersymmetric Grand Unified Theories, which can be made all-loop finite, both in the dimensionless (gauge and Yukawa couplings) and dimensionful (soft supersymmetry breaking terms) sectors. This…

We seek random versions of some classical theorems on complex approximation by polynomials and rational functions, as well as investigate properties of random compact sets in connection to complex approximation.

As NNs permeate various scientific and industrial domains, understanding the universality and reusability of their representations becomes crucial. At their core, these networks create intermediate neural representations, indicated as…

Neural operators, serving as physics surrogate models, have recently gained increased interest. With ever increasing problem complexity, the natural question arises: what is an efficient way to scale neural operators to larger and more…

As demonstrated in many areas of real-life applications, neural networks have the capability of dealing with high dimensional data. In the fields of optimal control and dynamical systems, the same capability was studied and verified in many…

In this chapter we take a look at the universal approximation question for stochastic feedforward neural networks. In contrast to deterministic networks, which represent mappings from a set of inputs to a set of outputs, stochastic networks…

The universal approximation property is fundamental to the success of neural networks, and has traditionally been achieved by training networks without any constraints on their parameters. However, recent experimental research proposed a…

The approximation power of general feedforward neural networks with piecewise linear activation functions is investigated. First, lower bounds on the size of a network are established in terms of the approximation error and network depth…

Constraining linear layers in neural networks to respect symmetry transformations from a group $G$ is a common design principle for invariant networks that has found many applications in machine learning. In this paper, we consider a…