Related papers: Kolmogorov Width Decay and Poor Approximators in M…

We compare the Kolmogorov and entropy numbers of compact operators mapping from a Hilbert space into a Banach space. We then apply these general findings to embeddings between reproducing kernel Hilbert spaces and $L_\infty(\mu)$. Here we…

Approximation processes in the reproducing kernel Hilbert space associated to a continuous kernel on the unit sphere $S^m$ in the Euclidean space $\mathbb{R}^{m+1}$ are known to depend upon the Mercer's expansion of the compact and…

We prove Carl's type inequalities for the error of approximation of compact sets K by deep and shallow neural networks. This in turn gives lower bounds on how well we can approximate the functions in K when requiring the approximants to…

This short note presents a linear algebraic approach to proving dimension lower bounds for linear methods that solve $L^2$ function approximation problems. The basic argument has appeared in the literature before (e.g., Barron, 1993) for…

If $L$ is a bounded linear operator mapping the Banach space $X$ into the Banach space $Y$ and $K$ is a compact set in $X$, then the Kolmogorov widths of the image $L(K)$ do not exceed those of $K$ multiplied by the norm of $L$. We extend…

This paper develops fundamental limits of deep neural network learning by characterizing what is possible if no constraints are imposed on the learning algorithm and on the amount of training data. Concretely, we consider Kolmogorov-optimal…

Low-rank approximation of kernels is a fundamental mathematical problem with widespread algorithmic applications. Often the kernel is restricted to an algebraic variety, e.g., in problems involving sparse or low-rank data. We show that…

Characterizing the function spaces defined by neural networks helps understanding the corresponding learning models and their inductive bias. While in some limits neural networks correspond to function spaces that are Hilbert spaces, these…

Multilayer Perceptrons (MLPs) have long been a cornerstone in deep learning, known for their capacity to model complex relationships. Recently, Kolmogorov-Arnold Networks (KANs) have emerged as a compelling alternative, utilizing highly…

We determine lower and exact estimates of Kolmogorov, Gelfand and linear $n$-widths of unit balls in Sobolev norms in $L_{p}$-spaces on compact Riemannian manifolds. As it was shown by us previously these lower estimates are exact…

This paper introduces a measure, called Lipschitz widths, of the optimal performance possible of certain nonlinear methods of approximation. It discusses their relation to entropy numbers and other well known widths such as the Kolmogorov…

This paper considers the following question: how well can depth-two ReLU networks with randomly initialized bottom-level weights represent smooth functions? We give near-matching upper- and lower-bounds for $L_2$-approximation in terms of…

We relate the problem of best low-rank approximation in the spectral norm for a matrix $A$ to Kolmogorov $n$-widths and corresponding optimal spaces. We characterize all the optimal spaces for the image of the Euclidean unit ball under $A$…

Kernel-based methods in Numerical Analysis have the advantage of yielding optimal recovery processes in the "native" Hilbert space $\calh$ in which they are reproducing. Continuous kernels on compact domains have an expansion into…

We consider the approximation rates of shallow neural networks with respect to the variation norm. Upper bounds on these rates have been established for sigmoidal and ReLU activation functions, but it has remained an important open problem…

Recent studies show that a reproducing kernel Hilbert space (RKHS) is not a suitable space to model functions by neural networks as the curse of dimensionality (CoD) cannot be evaded when trying to approximate even a single ReLU neuron…

We obtain exact lower bounds for Kolmogorov $n$-widths in spaces $C$ and $L$ of classes of convolutions with Neumann kernel $N_{q,\beta}(t)=\sum\limits_{k=1}^{\infty}\dfrac{q^k}{k}\cos\left(kt-\dfrac{\beta\pi}{2}\right)$, ${q\in(0,1)}$,…

We employ kernel-based approaches that use samples from a probability distribution to approximate a Kolmogorov operator on a manifold. The self-tuning variable-bandwidth kernel method [Berry & Harlim, Appl. Comput. Harmon. Anal.,…

Learning rates for least-squares regression are typically expressed in terms of $L_2$-norms. In this paper we extend these rates to norms stronger than the $L_2$-norm without requiring the regression function to be contained in the…

Kernel ridge regression is an important nonparametric method for estimating smooth functions. We introduce a new set of conditions, under which the actual rates of convergence of the kernel ridge regression estimator under both the L_2 norm…