Related papers: On the Saturation Effect of Kernel Ridge Regressio…

Let $\Pi_q$ be an arbitrary finite projective plane of order $q$. A subset $S$ of its points is called saturating if any point outside $S$ is collinear with a pair of points from $S$. Applying probabilistic tools we improve the upper bound…

Conditionally positive definite (CPD) kernels are defined with respect to a function class $\mathcal{F}$. It is well known that such a kernel $K$ is associated with its native space (defined analogously to an RKHS), which in turn gives rise…

I show that the MacColl-Hartman effect, namely, the saturation of the group delay time of sub-barrier quantum tunneling as a function of the barrier width, comes from the saturating behavior of a more fundamental concept - the phase of the…

Kernel ridge regression (KRR) is a foundational tool in machine learning, with recent work emphasizing its connections to neural networks. However, existing theory primarily addresses the i.i.d. setting, while real-world data often exhibits…

These notes are about ridge functions. Recent years have witnessed a flurry of interest in these functions. Ridge functions appear in various fields and under various guises. They appear in fields as diverse as partial differential…

This paper addresses the covariate shift problem in the context of nonparametric regression within reproducing kernel Hilbert spaces (RKHSs). Covariate shift arises in supervised learning when the input distributions of the training and…

The saturation mechanism for the fastest-growing instability of massive scalar field in Kerr metric is identified, assuming saturation by cubic or quartic nonlinearities of the field potential. The resulting spindown rate of the black hole…

In the absence of explicit regularization, Kernel "Ridgeless" Regression with nonlinear kernels has the potential to fit the training data perfectly. It has been observed empirically, however, that such interpolated solutions can still…

The classical kernel ridge regression problem aims to find the best fit for the output $Y$ as a function of the input data $X\in \mathbb{R}^d$, with a fixed choice of regularization term imposed by a given choice of a reproducing kernel…

Saturating sets are combinatorial objects in projective spaces over finite fields that have been intensively investigated in the last three decades. They are related to the so-called covering problem of codes in the Hamming metric. In this…

This article establishes sharp inverse and saturation statements for kernel-based approximation using finitely smooth Sobolev kernels on bounded Lipschitz regions. The analysis focuses on the superconvergence regime, for which direct…

We propose a framework for hypothesis testing on conditional probability distributions, which we then use to construct statistical tests of functionals of conditional distributions. These tests identify the inputs where the functionals…

We consider the problem of reconstructing a function from a finite set of noise-corrupted samples. Two kernel algorithms are analyzed, namely kernel ridge regression and $\varepsilon$-support vector regression. By assuming the ground-truth…

We develop and analyze a principled approach to kernel ridge regression under covariate shift. The goal is to learn a regression function with small mean squared error over a target distribution, based on unlabeled data from there and…

Relativistic heavy-ion collisions lead to a final state which has a higher degree of strangeness saturation than those of elementary collisions. A systematic analysis of this phenomenon, based on the strangeness saturation factor, is made…

We propose a metric, Layer Saturation, defined as the proportion of the number of eigenvalues needed to explain 99% of the variance of the latent representations, for analyzing the learned representations of neural network layers.…

Neural Collapse is a phenomenon that helps identify sparse and low rank structures in deep classifiers. Recent work has extended the definition of neural collapse to regression problems, albeit only measuring the phenomenon at the last…

A well-recognized limitation of kernel learning is the requirement to handle a kernel matrix, whose size is quadratic in the number of training examples. Many methods have been proposed to reduce this computational cost, mostly by using a…

Ridge estimation is an important manifold learning technique. The goal of this paper is to examine the effects of nonlinear transformations on the ridge sets. The main result proves the inclusion relationship between ridges: $\cR(f\circ…

This paper investigates the connection between neural networks and sufficient dimension reduction (SDR), demonstrating that neural networks inherently perform SDR in regression tasks under appropriate rank regularizations. Specifically, the…