Related papers: Stabilizing the Rayleigh--Ritz procedure by random…
We study the ridge regression (L2 regularized least squares) problem and its dual, which is also a ridge regression problem. We observe that the optimality conditions describing the primal and dual optimal solutions can be formulated in…
In this paper we present a convergence rate analysis of inexact variants of several randomized iterative methods. Among the methods studied are: stochastic gradient descent, stochastic Newton, stochastic proximal point and stochastic…
Calculation of the log-normalizer is a major computational obstacle in applications of log-linear models with large output spaces. The problem of fast normalizer computation has therefore attracted significant attention in the theoretical…
We investigate regularized algorithms combining with projection for least-squares regression problem over a Hilbert space, covering nonparametric regression over a reproducing kernel Hilbert space. We prove convergence results with respect…
Regular convergence, together with various other types of convergence, has been studied since the 1970s for the discrete approximations of linear operators. In this paper, we consider the eigenvalue approximation of compact operators whose…
We present two approximation methods for computing eigenfrequencies and eigenmodes of large-scale nonlinear eigenvalue problems resulting from boundary element method (BEM) solutions of some types of acoustic eigenvalue problems in…
Linear inverse problems are ubiquitous. Often the measurements do not follow a Gaussian distribution. Additionally, a model matrix with a large condition number can complicate the problem further by making it ill-posed. In this case, the…
We show that standard extragradient methods (i.e. mirror prox and dual extrapolation) recover optimal accelerated rates for first-order minimization of smooth convex functions. To obtain this result we provide a fine-grained…
We consider a stochastic version of the proximal point algorithm for optimization problems posed on a Hilbert space. A typical application of this is supervised learning. While the method is not new, it has not been extensively analyzed in…
Many popular eigensolvers for large and sparse Hermitian matrices or matrix pairs can be interpreted as accelerated block preconditioned gradient (BPG) iterations in order to analyze their convergence behavior by composing known estimates.…
Tomographic investigations are a central tool in medical applications, allowing doctors to image the interior of patients. The corresponding measurement process is commonly modeled by the Radon transform. In practice, the solution of the…
In this paper, we apply randomized algorithms to approximate the total least squares (TLS) solution of the problem $Ax\approx b$ in the large-scale discrete ill-posed problems. A regularization technique, based on the multiplicative…
Regularization is often used in high-dimensional regression settings to generate a sparse model, which can save tremendous computing resources and identify predictors that are most strongly associated with the response. When the predictors…
Consider the following three important problems in statistical inference, namely, constructing confidence intervals for (1) the error of a high-dimensional ($p>n$) regression estimator, (2) the linear regression noise level, and (3) the…
We describe a strategy for solving nonlinear eigenproblems numerically. Our approach is based on the approximation of a vector-valued function, defined as solution of a non-homogeneous version of the eigenproblem. This approximation step is…
We present a method for solving nonlinear eigenvalue problems using rational approximation. The method uses the AAA method by Nakatsukasa, S\`{e}te, and Trefethen to approximate the nonlinear eigenvalue problem by a rational eigenvalue…
Solving the generalized eigenvalue problem is a useful method for finding energy eigenstates of large quantum systems. It uses projection onto a set of basis states which are typically not orthogonal. One needs to invert a matrix whose…
The maximization of the (generalized) Rayleigh quotient is a central problem in numerical linear algebra. Conventional algorithms for its computation typically rely on matrix-adjoint products, making them sensitive to errors arising from…
In this paper, we introduce a randomized algorithm for solving the non-symmetric eigenvalue problem, referred to as randomized Implicitly Restarted Arnoldi (rIRA). This method relies on using a sketch-orthogonal basis during the Arnoldi…
The Rayleigh-Ritz procedure for determining bound-states of the Schr\"{o}dinger equation relies on spectral representation of the solution as a linear combination of the basis functions. Several possible extensions of the method to…