Related papers: Average-case thresholds for exact regularization o…
Modern regression problems often involve high-dimensional data and a careful tuning of the regularization hyperparameters is crucial to avoid overly complex models that may overfit the training data while guaranteeing desirable properties…
An open question contributed by Yu. Orlov to a recently published volume "Unsolved Problems in Mathematical Systems and Control Theory", V.D. Blondel, A. Megretski (eds), Princeton Univ. Press, 2004, concerns regularization of optimal…
Regularization is a popular technique to solve the overfitting problem of machine learning algorithms. Most regularization technique relies on parameter selection of the regularization coefficient. Plug-in method and cross-validation…
Length generalization is a key property of a learning algorithm that enables it to make correct predictions on inputs of any length, given finite training data. To provide such a guarantee, one needs to be able to compute a length…
To address the challenges of reliable statistical inference in high-dimensional models, we introduce the Synthetic-data Regularized Estimator (SRE). Unlike traditional regularization methods, the SRE regularizes the complex target model via…
This paper presents an achievability bound that evaluates the exact probability of error of an ensemble of random codes that are decoded by a minimum distance decoder. Compared to the state-of-the-art which demands exponential computation…
Many inverse problems can be described by a PDE model with unknown parameters that need to be calibrated based on measurements related to its solution. This can be seen as a constrained minimization problem where one wishes to minimize the…
We compute precise asymptotic expressions for the learning curves of least squares random feature (RF) models with either a separable strongly convex regularization or the $\ell_1$ regularization. We propose a novel multi-level application…
We introduce Stochastic Asymptotical Regularization (SAR) methods for the uncertainty quantification of the stable approximate solution of ill-posed linear-operator equations, which are deterministic models for numerous inverse problems in…
Let $X_1,\dots,X_n$ be i.i.d. log-concave random vectors in $\mathbb R^d$ with mean 0 and covariance matrix $\Sigma$. We study the problem of quantifying the normal approximation error for $W=n^{-1/2}\sum_{i=1}^nX_i$ with explicit…
Gaussian process regression (GPR) is a popular nonparametric Bayesian method that provides predictive uncertainty estimates and is widely used in safety-critical applications. While prior research has introduced various uncertainty bounds,…
Gaussian process regression is used throughout statistics and machine learning for prediction and uncertainty quantification. A Gaussian process is specified by its mean and covariance functions. Many covariance functions, including…
In this work, we consider the problem of regularization in the design of minimum mean square error (MMSE) linear filters. Using the relationship with statistical machine learning methods, using a Bayesian approach, the regularization…
Convex relaxations are effective for training and certifying neural networks against norm-bounded adversarial attacks, but they leave a large gap between certifiable and empirical robustness. In principle, convex relaxation can provide…
This article studies the achievable guarantees on the error rates of certain learning algorithms, with particular focus on refining logarithmic factors. Many of the results are based on a general technique for obtaining bounds on the error…
This paper considers regularizing a covariance matrix of $p$ variables estimated from $n$ observations, by hard thresholding. We show that the thresholded estimate is consistent in the operator norm as long as the true covariance matrix is…
We apply a probabilistic approach to study the computational complexity of analog computers which solve linear programming problems. We analyze numerically various ensembles of linear programming problems and obtain, for each of these…
Generalized equations are problems emerging in contexts of modern variational analysis as an adequate formalism to treat such issues as constraint systems, optimality and equilibrium conditions, variational inequalities, differential…
Data augmentation is one of the most popular techniques for improving the robustness of neural networks. In addition to directly training the model with original samples and augmented samples, a torrent of methods regularizing the distance…
The truncated singular value decomposition may be used to find the solution of linear discrete ill-posed problems in conjunction with Tikhonov regularization and requires the estimation of a regularization parameter that balances between…