Related papers: A universal approximation theorem and its applicat…

This paper develops simple feed-forward neural networks that achieve the universal approximation property for all continuous functions with a fixed finite number of neurons. These neural networks are simple because they are designed with a…

The theory of universal Taylor series can be extended to the case of Pad\'e approximants where the universal approximation is not realized by polynomials any more, but by rational functions, namely the Pad\'e approximants of some power…

The paper proposes an artificial neural network (ANN) being a global approximator for a special class of functions, which are known as generalized homogeneous. The homogeneity means a symmetry of a function with respect to a group of…

We show that there are no non-trivial closed subspaces of $L_2(\mathbb{R}^n)$ that are invariant under invertible affine transformations. We apply this result to neural networks showing that any nonzero $L_2(\mathbb{R})$ function is an…

It is a classical result in rational approximation theory that certain non-smooth or singular functions, such as $|x|$ and $x^{1/p}$, can be efficiently approximated using rational functions with root-exponential convergence in terms of…

Training neural networks to be certifiably robust is critical to ensure their safety against adversarial attacks. However, it is currently very difficult to train a neural network that is both accurate and certifiably robust. In this work…

Group symmetry is inherent in a wide variety of data distributions. Data processing that preserves symmetry is described as an equivariant map and often effective in achieving high performance. Convolutional neural networks (CNNs) have been…

Let us assume that $f$ is a continuous function defined on the unit ball of $\mathbb R^d$, of the form $f(x) = g (A x)$, where $A$ is a $k \times d$ matrix and $g$ is a function of $k$ variables for $k \ll d$. We are given a budget $m \in…

The study of universal approximation of arbitrary functions $f: \mathcal{X} \to \mathcal{Y}$ by neural networks has a rich and thorough history dating back to Kolmogorov (1957). In the case of learning finite dimensional maps, many authors…

Let $f$ be a measurable, real function defined in a neighbourhood of infinity. The function $f$ is said to be of generalised regular variation if there exist functions $h \not\equiv 0$ and $g > 0$ such that $f(xt) - f(t) = h(x) g(t) +…

We present an approximation theorem for continuous non-decreasing functions on compact preordered spaces, leading to an algebraic characterization of their corresponding function spaces. As an application, we prove that the family of…

We give an abstract approach to approximations with a wide range of regularity classes $X$ in spaces of pseudocontinuable functions $K^p_\vartheta$, where $\vartheta$ is an inner function and $p>0$. More precisely, we demonstrate a general…

There are several kinds of universal Taylor series. In one such kind the universal approximation is required at every boundary point of the domain of definition $\OO$ of the universal function $f$. In another kind the universal…

We propose a testable universality hypothesis, asserting that seemingly disparate neural network solutions observed in the simple task of modular addition are unified under a common abstract algorithm. While prior work interpreted…

The celebrated universal approximation theorems for neural networks roughly state that any reasonable function can be arbitrarily well-approximated by a network whose parameters are appropriately chosen real numbers. This paper examines the…

We study feedforward neural networks with inputs from a topological space (TFNNs). We prove a universal approximation theorem for shallow TFNNs, which demonstrates their capacity to approximate any continuous function defined on this…

In this short note, we give an elementary proof of a universal approximation theorem for neural networks with three hidden layers and increasing, continuous, bounded activation function. The result is weaker than the best known results, but…

According to a classical result of Szemer\'{e}di, every dense subset of $1,2,...,N$ contains an arbitrary long arithmetic progression, if $N$ is large enough. Its analogue in higher dimensions due to F\"urstenberg and Katznelson says that…

In this paper, we explain the universal approximation capabilities of deep residual neural networks through geometric nonlinear control. Inspired by recent work establishing links between residual networks and control systems, we provide a…

Density functional theory is a successful branch of numerical simulations of quantum systems. While the foundations are rigorously defined, the universal functional must be approximated resulting in a `semi'-ab initio approach. The search…