Related papers: Estimation and inference in error-in-operator mode…

Tikhonov regularization is a common technique used when solving poorly behaved optimization problems. Often, and with good reason, this technique is applied by practitioners in an ad hoc fashion. In this note, we systematically illustrate…

Induction benefits from useful priors. Penalized regression approaches, like ridge regression, shrink weights toward zero but zero association is usually not a sensible prior. Inspired by simple and robust decision heuristics humans use, we…

Orthogonality constraints naturally appear in many machine learning problems, from principal component analysis to robust neural network training. They are usually solved using Riemannian optimization algorithms, which minimize the…

We study the stability of regularization by projection for solving linear inverse problems if the forward operator is given indirectly but specified via some input-output training pairs. We extend the approach in "Data driven regularization…

This paper develops a discrete data-driven approach for solving the inverse source problem of the wave equation with final time measurements. Focusing on the $L^2$-Tikhonov regularization method, we analyze its convergence under two…

Inverse problems are common and important in many applications in computational physics but are inherently ill-posed with many possible model parameters resulting in satisfactory results in the observation space. When solving the inverse…

Multiple model reduction techniques have been proposed to tackle linear and non linear problems. Intrusive model order reduction techniques exhibit high accuracy levels, however, they are rarely used as a standalone industrial tool, because…

We consider the problem of online linear regression in the stochastic setting. We derive high probability regret bounds for online ridge regression and the forward algorithm. This enables us to compare online regression algorithms more…

For an ill-posed inverse problem, particularly with incomplete and limited measurement data, regularization is an essential tool for stabilizing the inverse problem. Among various forms of regularization, the lp penalty term provides a…

We consider the sequential decision-making problem where the mean outcome is a non-linear function of the chosen action. Compared with the linear model, two curious phenomena arise in non-linear models: first, in addition to the "learning…

Maximum likelihood estimation in nonlinear models can exhibit substantial instability in finite samples when the data provide limited information about certain parameters. Such instability is driven by rare but extreme realizations of the…

We construct an efficient numerical scheme for solving obstacle problems in divergence form. The numerical method is based on a reformulation of the obstacle in terms of an L1-like penalty on the variational problem. The reformulation is an…

Motivated by the EU's "Right To Be Forgotten" regulation, we initiate a study of statistical data deletion problems where users' data are accessible only for a limited period of time. This setting is formulated as an online supervised…

This paper studies sparse linear regression analysis with outliers in the responses. A parameter vector for modeling outliers is added to the standard linear regression model and then the sparse estimation problem for both coefficients and…

We propose an approach to reduce the bias of ridge regression and regularization kernel network. When applied to a single data set the new algorithms have comparable learning performance with the original ones. When applied to incremental…

Iterative regularization is a classic idea in regularization theory, that has recently become popular in machine learning. On the one hand, it allows to design efficient algorithms controlling at the same time numerical and statistical…

We connect stochastic resetting from non-equilibrium statistical physics with ridge regularization in statistical learning. For linear gradient flow, resetting to the origin at rate $r$ produces stationary mean $(X^\top X+rI)^{-1}X^\top y$,…

Fitting linear regression models can be computationally very expensive in large-scale data analysis tasks if the sample size and the number of variables are very large. Random projections are extensively used as a dimension reduction tool…

Nonlinear parametric inverse problems appear in many applications and are typically very expensive to solve, especially if they involve many measurements. These problems pose huge computational challenges as evaluating the objective…

Tikhonov regularization for projected solutions of large-scale ill-posed problems is considered. The Golub-Kahan iterative bidiagonalization is used to project the problem onto a subspace and regularization then applied to find a subspace…