Related papers: Schauder Bases for $C[0, 1]$ Using ReLU, Softplus …

Submodular functions are a fundamental object of study in combinatorial optimization, economics, machine learning, etc. and exhibit a rich combinatorial structure. Many subclasses of submodular functions have also been well studied and…

Let $U\subseteq\mathbb{R}^{n}$ be open and convex. We show that every (not necessarily Lipschitz or strongly) convex function $f:U\to\mathbb{R}$ can be approximated by real analytic convex functions, uniformly on all of $U$. In doing so we…

The Rectified Linear Unit (ReLU) is a foundational activation function in artficial neural networks. Recent literature frequently misattributes its origin to the 2018 (initial) version of this paper, which exclusively investigated ReLU at…

We develop a corrective mechanism for neural network approximation: the total available non-linear units are divided into multiple groups and the first group approximates the function under consideration, the second group approximates the…

This article concerns the expressive power of depth in deep feed-forward neural nets with ReLU activations. Specifically, we answer the following question: for a fixed $d_{in}\geq 1,$ what is the minimal width $w$ so that neural nets with…

This paper introduces a novel framework for constructing $C^r$ basis functions for polynomial spline spaces of degree $d$ over arbitrary planar polygonal partitions, overturning the belief that basis functions cannot be constructed on…

Universal approximation theorems show that neural networks can approximate any continuous function; however, the number of parameters may grow exponentially with the ambient dimension, so these results do not fully explain the practical…

We consider the Lommel functions $s_{\mu,\nu}(z)$ for different values of the parameters $(\mu,\nu)$. We show that if $(\mu,\nu)$ are half integers, then it is possible to describe these functions with an explicit combination of polynomials…

We investigate the existence of equivalent p-norms, 0< p 1, under which conditional symmetric or spreading bases in quasi-Banach spaces become isometric. For spreading bases (which need not be unconditional or even Schauder bases), we…

A key element of understanding the efficacy of overparameterized neural networks is characterizing how they represent functions as the number of weights in the network approaches infinity. In this paper, we characterize the norm required to…

When studying the expressive power of neural networks, a main challenge is to understand how the size and depth of the network affect its ability to approximate real functions. However, not all functions are interesting from a practical…

Haglund, Luoto, Mason, and van Willigenburg introduced a basis for quasisymmetric functions called the quasisymmetric Schur function basis, generated combinatorially through fillings of composition diagrams in much the same way as Schur…

Motivated by problems on Brownian motion, we introduce a recursive scheme for a basis construction in the Hilbert space L^2(0,1) which is analogous to that of Haar and Walsh. More generally, we find a new decomposition theory for the…

Let $\mathcal{H}$ be a complex separable Hilbert space. We prove that if $\{f_{n}\}_{n=1}^{\infty}$ is a Schauder basis of the Hilbert space $\mathcal{H}$, then the angles between any two vectors in this basis must have a positive lower…

We study ReLU deep neural networks (DNNs) by investigating their connections with the hierarchical basis method in finite element methods. First, we show that the approximation schemes of ReLU DNNs for $x^2$ and $xy$ are composition…

In this document I develop a weight function theory of positive order basis function interpolants and smoothers. **In Chapter 1 the basis functions and data spaces are defined directly using weight functions. The data spaces are used to…

We present an orthogonal basis for functions over a slice of the Boolean hypercube. Our basis is also an orthogonal basis of eigenvectors for the Johnson and Kneser graphs. As an application of our basis, we streamline Wimmer's proof of…

We investigate properties of neural networks that use both ReLU and $x^2$ as activation functions and build upon previous results to show that both analytic functions and functions in Sobolev spaces can be approximated by such networks of…

In this paper, we discuss the problem of constructing Radial Basis In this paper, we discuss the problem of constructing Radial Basis Function (RBF)-based Partition of Unity (PU) interpolants that are positive if data values are positive.…

This paper studies the approximation property of ReLU neural networks (NNs) to piecewise constant functions with unknown interfaces in bounded regions in $\mathbb{R}^d$. Under the assumption that the discontinuity interface $\Gamma$ may be…