Related papers: Understanding Kernel Ridge Regression: Common beha…

With the rise of large language models (LLMs) for flexibly processing information as strings, a natural application is regression, specifically by preprocessing string representations into LLM embeddings as downstream features for metric…

The random feature (RF) approach is a well-established and efficient tool for scalable kernel methods, but existing literature has primarily focused on kernel ridge regression with random features (KRR-RF), which has limitations in handling…

Randomized neural networks (NNs) are an interesting alternative to conventional NNs that are more used for data modeling. The random vector functional-link (RVFL) network is an established and theoretically well-grounded randomized learning…

We propose new reproducing kernel-based tests for model checking in conditional moment restriction models. By regressing estimated residuals on kernel functions via kernel ridge regression (KRR), we obtain a coefficient function in a…

Machine learning (ML) methods are highly flexible, but their ability to approximate the true data-generating process is fundamentally constrained by finite samples. We characterize a universal lower bound, the Limits-to-Learning Gap (LLG),…

Machine Learning (ML) is increasingly being used for computer aided diagnosis of brain related disorders based on structural magnetic resonance imaging (MRI) data. Most of such work employs biologically and medically meaningful hand-crafted…

Despite their many appealing properties, kernel methods are heavily affected by the curse of dimensionality. For instance, in the case of inner product kernels in $\mathbb{R}^d$, the Reproducing Kernel Hilbert Space (RKHS) norm is often…

We develop a framework for function classes generated by parametric ridge kernels: one-dimensional kernels composed with affine projections and averaged over a parameter measure. The induced kernels are positive definite, and the resulting…

Semisupervised methods inevitably invoke some assumption that links the marginal distribution of the features to the regression function of the label. Most commonly, the cluster or manifold assumptions are used which imply that the…

In this technical report, we explore the behavior of Recursive Feature Machines (RFMs), a type of novel kernel machine that recursively learns features via the average gradient outer product, through a series of experiments on regression…

Machine Learning (ML) is an expressive framework for turning data into computer programs. Across many problem domains -- both in industry and policy settings -- the types of computer programs needed for accurate prediction or optimal…

Estimating the innovation probability density is an important issue in any regression analysis. This paper focuses on functional autoregressive models. A residual-based kernel estimator is proposed for the innovation density. Asymptotic…

We explore the application of computer vision and machine learning (ML) techniques to predict material properties (e.g. compressive strength) based on SEM images. We show that it's possible to train ML models to predict materials…

Kernel density estimation is a widely used nonparametric approach to estimate an unknown distribution. Recent work in Bayesian predictive inference has considered stochastic processes formed by specifying the predictive distribution for the…

Kernel Ridge Regression (KRR) is a simple yet powerful technique for non-parametric regression whose computation amounts to solving a linear system. This system is usually dense and highly ill-conditioned. In addition, the dimensions of the…

Density functional theory is the standard theory for computing the electronic structure of materials, which is based on a functional that maps the electron density to the energy. However, a rigorous form of the functional is not known and…

Work in machine learning and statistics commonly focuses on building models that capture the vast majority of data, possibly ignoring a segment of the population as outliers. However, there does not often exist a good model on the whole…

The formally exact framework of equilibrium Density Functional Theory (DFT) is capable of simultaneously and consistently describing thermodynamic and structural properties of interacting many-body systems in arbitrary external potentials.…

We propose a doubly robust inference method for causal effects of continuous treatment variables, under unconfoundedness and with nonparametric or high-dimensional nuisance functions. Our double debiased machine learning (DML) estimators…

We explore unique considerations involved in fitting ML models to data with very high precision, as is often required for science applications. We empirically compare various function approximation methods and study how they scale with…