Related papers: Sobolev-type embeddings for neural network approxi…

Embeddings are a basic initial feature extraction step in many machine learning models, particularly in natural language processing. An embedding attempts to map data tokens to a low-dimensional space where similar tokens are mapped to…

Neural networks are one of the most popularly used methods in machine learning and artificial intelligence nowadays. Due to the universal approximation theorem (Hornik et al. (1989)), a neural network with one hidden layer can approximate…

This paper considers the following question: how well can depth-two ReLU networks with randomly initialized bottom-level weights represent smooth functions? We give near-matching upper- and lower-bounds for $L_2$-approximation in terms of…

In this article, via certain lower bound conditions on the measures under consideration, the authors fully characterize the Sobolev embeddings for the scales of Haj{\l}asz-Triebel-Lizorkin and Haj{\l}asz-Besov spaces in the general context…

A very popular class of models for networks posits that each node is represented by a point in a continuous latent space, and that the probability of an edge between nodes is a decreasing function of the distance between them in this latent…

Convolution blocks serve as local feature extractors and are the key to success of the neural networks. To make local semantic feature embedding rather explicit, we reformulate convolution blocks as feature selection according to the best…

Prototypical network for Few shot learning tries to learn an embedding function in the encoder that embeds images with similar features close to one another in the embedding space. However, in this process, the support set samples for a…

While the universal approximation property holds both for hierarchical and shallow networks, we prove that deep (hierarchical) networks can approximate the class of compositional functions with the same accuracy as shallow networks but with…

We prove that for every Banach space $Y$, the Besov spaces of functions from the $n$-dimensional Euclidean space to $Y$ agree with suitable local approximation spaces with equivalent norms. In addition, we prove that the Sobolev spaces of…

Using tools from the theory of operator ideals and s-numbers, we develop a general approach to transfer estimates for $L_2$ -approximation of Sobolev functions into estimates for $L_\infty$-approximation, with precise control of all…

We provide a complete characterization of compactness of Sobolev embeddings of radially symmetric functions on the entire space $\mathbb{R}^n$ in the general framework of rearrangement-invariant function spaces. We avoid any unnecessary…

A common technique for compressing a neural network is to compute the $k$-rank $\ell_2$ approximation $A_{k,2}$ of the matrix $A\in\mathbb{R}^{n\times d}$ that corresponds to a fully connected layer (or embedding layer). Here, $d$ is the…

We study the problem of estimating an unknown function from noisy data using shallow ReLU neural networks. The estimators we study minimize the sum of squared data-fitting errors plus a regularization term proportional to the squared…

The celebrated universal approximation theorems for neural networks roughly state that any reasonable function can be arbitrarily well-approximated by a network whose parameters are appropriately chosen real numbers. This paper examines the…

We consider in this paper the optimal approximations of convex univariate functions with feed-forward Relu neural networks. We are interested in the following question: what is the minimal approximation error given the number of…

Structural network embedding is a crucial step in enabling effective downstream tasks for complex systems that aims to project a network into a lower-dimensional space while preserving similarities among nodes. We introduce a simple and…

We prove that almost every level set of a Sobolev function in a planar domain consists of points, Jordan curves, or homeomorphic copies of an interval. For monotone Sobolev functions in the plane we have the stronger conclusion that almost…

We give an intrinsic characterization of the restrictions of Sobolev, Triebel-Lizorkin and Besov spaces to regular subsets of $R^n$ via sharp maximal functions and local approximations.

In 1989 George Cybenko proved in a landmark paper that wide shallow neural networks can approximate arbitrary continuous functions on a compact set. This universal approximation theorem sparked a lot of follow-up research. Shen, Yang and…

We propose a functional framework of fractional Sobolev spaces for a class of ultra-parabolic Kolmogorov type operators satisfying the weak H\"ormander condition. We characterize these spaces as real interpolation of natural order intrinic…