Related papers: Schauder Bases for $C[0, 1]$ Using ReLU, Softplus …
The paper briefy reviews several recent results on hierarchical architectures for learning from examples, that may formally explain the conditions under which Deep Convolutional Neural Networks perform much better in function approximation…
A well-known line of work (Barron, 1993; Breiman, 1993; Klusowski & Barron, 2018) provides bounds on the width $n$ of a ReLU two-layer neural network needed to approximate a function $f$ over the ball $\mathcal{B}_R(\mathbb{R}^d)$ up to…
Large language models (LLMs), such as ChatGPT and GPT4, have shown outstanding performance in many human life task. Attention computation plays an important role in training LLMs. Softmax unit and ReLU unit are the key structure in…
We discuss avoidance of sure loss and coherence results for semicopulas and standardized functions, i.e., for grounded, 1-increasing functions with value $1$ at $(1,1,\ldots, 1)$. We characterize the existence of a $k$-increasing…
The paper considers the Hilbert space $\hat{H}_r$ of real functions summable with the square $L^2(a,b)_r$ on any interval $\{(a,b)_r\}_{r=1}^{\infty}\in \mathbb{R}$. It is shown on the basis of the theorem on zeros of real orthogonal…
In this paper, we prove the following results. There exists a Banach space without basis which has a Schauder frame. There exists an universal Banach space $B$ (resp. $\tilde{B}$) with a basis (resp. an unconditional basis) such that, a…
The reconstruction theorem and the multilevel Schauder estimate have central roles in the analytic theory of regularity structures [17]. Inspired by [26], we provide elementary proofs for them by using the semigroup of operators.…
We consider the problem of approximating smoothing spline estimators in a nonparametric regression model. When applied to a sample of size $n$, the smoothing spline estimator can be expressed as a linear combination of $n$ basis functions,…
We establish a poset structure on combinatorial bases of multivariate polynomials defined by positive expansions, and study properties common to bases in this poset. Included are the well-studied bases of Schubert polynomials, Demazure…
It is shown that over-parameterized neural networks can achieve minimax optimal rates of convergence (up to logarithmic factors) for learning functions from certain smooth function classes, if the weights are suitably constrained or…
Neural networks are regularly employed in adaptive control of nonlinear systems and related methods of reinforcement learning. A common architecture uses a neural network with a single hidden layer (i.e. a shallow network), in which the…
We study universal approximation of continuous functionals on compact subsets of products of Hilbert spaces. We prove that any such functional can be uniformly approximated by models that first take finitely many continuous linear…
We introduce a proximal subdifferential and develop a calculus for nonsmooth functions defined on any Riemannian manifold $M$. We give several applications of this theory, concerning: 1) differentiability and geometrical properties of the…
Adapting the recently developed randomized dyadic structures, we introduce the notion of spline function in geometrically doubling quasi-metric spaces. Such functions have interpolation and reproducing properties as the linear splines in…
Recently there has been much interest in understanding why deep neural networks are preferred to shallow networks. We show that, for a large class of piecewise smooth functions, the number of neurons needed by a shallow network to…
We use a method, first developed for the Riemann zeta-function by Masser in ["Rational values of the Riemann zeta function", Journ. Num. Th. 131 (2011), 2037-2046], to prove a new zero estimate for polynomials in z and 1/Gamma(z). This…
We show how Lasry-Lions's result on regularization of functions defined on $\mathbb{R}^n$ or on Hilbert spaces by sup-inf convolutions with squares of distances can be extended to (finite or infinite dimensional) Riemannian manifolds $M$ of…
In this paper we develop the theory of Schauder estimates for the fractional harmonic oscillator $H^\sigma=(-\Delta+|x|^2)^\sigma$, $0<\sigma<1$. More precisely, a new class of smooth functions $C^{k,\alpha}_H$ is defined, in which we study…
This paper studies the problem of how efficiently functions in the Sobolev spaces $\mathcal{W}^{s,q}([0,1]^d)$ and Besov spaces $\mathcal{B}^s_{q,r}([0,1]^d)$ can be approximated by deep ReLU neural networks with width $W$ and depth $L$,…
A basilar property and a useful tool in the theory of Sobolev spaces is the density of smooth compactly supported functions in the space $W^{k,p}(\R^n)$ (i.e. the functions with weak derivatives of orders $0$ to $k$ in $L^p$). On Riemannian…