Related papers: $\mathcal{C}^1$-approximation with rational functi…

Neural networks are widely used to approximate unknown functions in control. A common neural network architecture uses a single hidden layer (i.e. a shallow network), in which the input parameters are fixed in advance and only the output…

We investigate random compact sets with random functions defined thereon, such as polynomials, rational functions, the pluricomplex Green function and the Siciak extremal function. One surprising consequence of our study is that randomness…

In 1989 George Cybenko proved in a landmark paper that wide shallow neural networks can approximate arbitrary continuous functions on a compact set. This universal approximation theorem sparked a lot of follow-up research. Shen, Yang and…

This paper studies the approximation capacity of neural networks with an arbitrary activation function and with norm constraint on the weights. Upper and lower bounds on the approximation error of these networks are computed for smooth…

We present a new rational approximation algorithm based on the empirical interpolation method for interpolating a family of parametrized functions to rational polynomials with invariant poles, leading to efficient numerical algorithms for…

For a real number $x$, call $\frac1n \lfloor nx \rfloor$ the $n$-th lower rational approximation of $x$. We study the functions defined by taking the cumulative average of the first $n$ lower rational approximations of $x$, which we call…

In this paper, we demonstrate the application of generalised rational uniform (Chebyshev) approximation in neural networks. In particular, our activation functions are one degree rational functions and the loss function is based on the…

An exponential-type approximation of the first order Marcum $Q$-function is presented, which is robust to changes in its first argument and can easily be integrated with respect to the second argument. Such characteristics are particularly…

Universal approximation theorems show that neural networks can approximate any continuous function; however, the number of parameters may grow exponentially with the ambient dimension, so these results do not fully explain the practical…

Functions with singularities are notoriously difficult to approximate with conventional approximation schemes. In computational applications, they are often resolved with low-order piecewise polynomials, multilevel schemes, or other types…

We consider the problem of approximating flow functions of continuous-time dynamical systems with inputs. It is well-known that continuous-time recurrent neural networks are universal approximators of this type of system. In this paper, we…

The study of universal approximation of arbitrary functions $f: \mathcal{X} \to \mathcal{Y}$ by neural networks has a rich and thorough history dating back to Kolmogorov (1957). In the case of learning finite dimensional maps, many authors…

This paper studies the approximation capabilities of neural networks that combine layer normalization (LN) with linear layers. We prove that networks consisting of two linear layers with parallel layer normalizations (PLNs) inserted between…

We study the extent to which wide neural networks may be approximated by Gaussian processes when initialized with random weights. It is a well-established fact that as the width of a network goes to infinity, its law converges to that of a…

We introduce a class of fully-connected neural networks whose activation functions, rather than being pointwise, rescale feature vectors by a function depending only on their norm. We call such networks radial neural networks, extending…

In this paper, we investigate the approximation behavior of both one and multidimensional neural network type operators for functions in $L^p(I^d,\rho)$, where $1\leq p<\infty$, associated with a general measure $\rho$ defined over a…

This paper concentrates on the approximation power of deep feed-forward neural networks in terms of width and depth. It is proved by construction that ReLU networks with width $\mathcal{O}\big(\max\{d\lfloor N^{1/d}\rfloor,\, N+2\}\big)$…

The foundations of deep learning are supported by the seemingly opposing perspectives of approximation or learning theory. The former advocates for large/expressive models that need not generalize, while the latter considers classes that…

Recently there has been much interest in understanding why deep neural networks are preferred to shallow networks. We show that, for a large class of piecewise smooth functions, the number of neurons needed by a shallow network to…

A common lens to theoretically study neural net architectures is to analyze the functions they can approximate. However, constructions from approximation theory may be unrealistic and therefore less meaningful. For example, a common…