Related papers: A Block Bidiagonalization Method for Fixed-Accurac…
The Lanczos algorithm has proven itself to be a valuable matrix eigensolver for problems with large dimensions, up to hundreds of millions or even tens of billions. The computational cost of using any Lanczos algorithm is dominated by the…
A rank-$r$ matrix $X \in \mathbb{R}^{m \times n}$ can be written as a product $U V^\top$, where $U \in \mathbb{R}^{m \times r}$ and $V \in \mathbb{R}^{n \times r}$. One could exploit this observation in optimization: e.g., consider the…
We study the problem of minimizing the sum of three convex functions: a differentiable, twice-differentiable and a non-smooth term in a high dimensional setting. To this effect we propose and analyze a randomized block cubic Newton (RBCN)…
The regularized Barzilai-Borwein (RBB) method represents a promising gradient-based optimization algorithm. In this paper, by splitting the gradient into two parts and analyzing the dynamical system of difference equations governing the…
Several recent randomized linear algebra algorithms rely upon fast dimension reduction methods. A popular choice is the Subsampled Randomized Hadamard Transform (SRHT). In this article, we address the efficacy, in the Frobenius and spectral…
Standard rank-revealing factorizations such as the singular value decomposition and column pivoted QR factorization are challenging to implement efficiently on a GPU. A major difficulty in this regard is the inability of standard algorithms…
We describe an algorithm for sampling a low-rank random matrix $Q$ that best approximates a fixed target matrix $P\in\mathbb{C}^{n\times m}$ in the following sense: $Q$ is unbiased, i.e., $\mathbb{E}[Q] = P$; $\mathsf{rank}(Q)\leq r$; and…
The problem of low rank approximation is ubiquitous in science. Traditionally this problem is solved in unitary invariant norms such as Frobenius or spectral norm due to existence of efficient methods for building approximations. However,…
Block-coordinate algorithms are recognized to furnish efficient iterative schemes for addressing large-scale problems, especially when the computation of full derivatives entails substantial memory requirements and computational efforts. In…
Randomized iterative algorithms have recently been proposed to solve large-scale linear systems. In this paper, we present a simple randomized extended block Kaczmarz algorithm that exponentially converges in the mean square to the unique…
Black-box variational inference (BBVI) now sees widespread use in machine learning and statistics as a fast yet flexible alternative to Markov chain Monte Carlo methods for approximate Bayesian inference. However, stochastic optimization…
In this paper, we present a new approach for model reduction of large scale first and second order dynamical systems with multiple inputs and multiple outputs (MIMO). This approach is based on the projection of the initial problem onto…
Randomized block Krylov subspace methods form a powerful class of algorithms for computing the extreme eigenvalues of a symmetric matrix or the extreme singular values of a general matrix. The purpose of this paper is to develop new…
Many applications in scientific computing and data science require the computation of a rank-revealing factorization of a large matrix. In many of these instances the classical algorithms for computing the singular value decomposition are…
We study the problem of selecting the shape parameter in Radial Basis function (RBF) interpolation using leave-one-out-cross-validation (LOOCV). Since the classical LOOCV formula requires repeated solves with a dense $N \times N$ kernel…
Low-rank approximations are popular methods to reduce the high computational cost of algorithms involving large-scale kernel matrices. The success of low-rank methods hinges on the matrix rank of the kernel matrix, and in practice, these…
Lanczos-type algorithms are efficient and easy to implement. Unfortunately they breakdown frequently and well before convergence has been achieved. These algorithms are typically based on recurrence relations which involve formal orthogonal…
In theory, the Lanczos algorithm generates an orthogonal basis of the corresponding Krylov subspace. However, in finite precision arithmetic, the orthogonality and linear independence of the computed Lanczos vectors is usually lost quickly.…
Structured Low-Rank Approximation is a problem arising in a wide range of applications in Numerical Analysis and Engineering Sciences. Given an input matrix $M$, the goal is to compute a matrix $M'$ of given rank $r$ in a linear or affine…
Low-rank matrix approximation play a ubiquitous role in various applications such as image processing, signal processing, and data analysis. Recently, random algorithms of low-rank matrix approximation have gained widespread adoption due to…