Related papers: Sobolev-type embeddings for neural network approxi…
We present an information-theoretic framework to learn fixed-dimensional embeddings for tasks in reinforcement learning. We leverage the idea that two tasks are similar if observing an agent's performance on one task reduces our uncertainty…
Based on the tree architecture, the objective of this paper is to design deep neural networks with two or more hidden layers (called deep nets) for realization of radial functions so as to enable rotational invariance for near-optimal…
We provide several new depth-based separation results for feed-forward neural networks, proving that various types of simple and natural functions can be better approximated using deeper networks than shallower ones, even if the shallower…
In this paper, we prove the rate of approximation for the Neural Network Sampling Operators activated by sigmoidal functions with mixed Lebesgue norm in terms of averaged modulus of smoothness for a bounded measurable functions on bounded…
A key element of understanding the efficacy of overparameterized neural networks is characterizing how they represent functions as the number of weights in the network approaches infinity. In this paper, we characterize the norm required to…
Deep learning has exhibited superior performance for various tasks, especially for high-dimensional datasets, such as images. To understand this property, we investigate the approximation and estimation ability of deep learning on…
We consider the problem of approximating a given element $f$ from a Hilbert space $\mathcal{H}$ by means of greedy algorithms and the application of such procedures to the regression problem in statistical learning theory. We improve on the…
We study shallow and deep neural networks whose inputs range over a general topological space. The model is built from a prescribed family of continuous feature maps and reduces to multilayer feedforward networks in the Euclidean case. We…
In the present paper we study embedding operators for weighted Sobolev spaces whose weights satisfy the well-known Muckenhoupt A_p-condition. Sufficient conditions for boundedness and compactness of the embedding operators are obtained for…
Universal approximation theorem suggests that a shallow neural network can approximate any function. The input to neurons at each layer is a weighted sum of previous layer neurons and then an activation is applied. These activation…
Motivated by the abundance of functional data such as time series and images, there has been a growing interest in integrating such data into neural networks and learning maps from function spaces to R (i.e., functionals). In this paper, we…
Deep learning models for human activity recognition (HAR) based on sensor data have been heavily studied recently. However, the generalization ability of deep models on complex real-world HAR data is limited by the availability of…
We consider the imbedding inequality || f ||_{L^r(R^d)} <= S_{r,n,d} || f ||_{H^{n}(R^d)}; H^{n}(R^d) is the Sobolev space (or Bessel potential space) of L^2 type and (integer or fractional) order n. We write down upper bounds for the…
This paper establishes the (nearly) optimal approximation error characterization of deep rectified linear unit (ReLU) networks for smooth functions in terms of both width and depth simultaneously. To that end, we first prove that…
On a general open set of the euclidean space, we study the relation between the embedding of the homogeneous Sobolev space $\mathcal{D}^{1,p}_0$ into $L^q$ and the summability properties of the distance function. We prove that in the…
Deep neural networks have been widely used as universal approximators for functions with inherent physical structures, including permutation symmetry. In this paper, we construct symmetric deep neural networks to approximate symmetric…
We introduce intrinsic Sobolev-Slobodeckij spaces for a class of ultra-parabolic Kolmogorov type operators satisfying the weak H\"ormander condition. We prove continuous embeddings into Lorentz and intrinsic H\"older spaces. We also prove…
While deep learning is successful in a number of applications, it is not yet well understood theoretically. A satisfactory theoretical characterization of deep learning however, is beginning to emerge. It covers the following questions: 1)…
In this article, we study approximation properties of the variation spaces corresponding to shallow neural networks with a variety of activation functions. We introduce two main tools for estimating the metric entropy, approximation rates,…
The article deals with a simplified proof of the Sobolev embedding theorem for Lizorkin--Triebel spaces (that contain the $L_p$-Sobolev spaces $H^s_p$ as special cases). The method extends to a proof of the corresponding fact for general…