Related papers: Computing accurate singular values using a mixed-p…
The randomized singular value decomposition (SVD) is a popular and effective algorithm for computing a near-best rank $k$ approximation of a matrix $A$ using matrix-vector products with standard Gaussian vectors. Here, we generalize the…
Higher-order singular value decomposition (HOSVD) is an efficient way for data reduction and also eliciting intrinsic structure of multi-dimensional array data. It has been used in many applications, and some of them involve incomplete…
Jacobi-type algorithms for simultaneous approximate diagonalization of real (or complex) symmetric tensors have been widely used in independent component analysis (ICA) because of their good performance. One natural way of choosing the…
Most nonlinear partial differential equation (PDE) solvers require the Jacobian matrix associated to the differential operator. In PETSc, this is typically achieved by either an analytic derivation or numerical approximation method such as…
When $k$ is a field, the classical Jacobian criterion computes the singular locus of an equidimensional, finitely generated $k$-algebra as the closed subset of an ideal generated by appropriate minors of the so-called Jacobian matrix.…
Motivated by the increasing availability of high-performance parallel computing, we design a distributed parallel algorithm for linearly-coupled block-structured nonconvex constrained optimization problems. Our algorithm performs…
Preconditioning is essential in iterative methods for solving linear systems. It is also the implicit objective in updating approximations of Jacobians in optimization methods, e.g.,in quasi-Newton methods. Motivated by the latter, we study…
In this work we introduce methods to reduce the computational and memory costs of training deep neural networks. Our approach consists in replacing exact vector-jacobian products by randomized, unbiased approximations thereof during…
The Newton, Gauss--Newton and Levenberg--Marquardt methods all use the first derivative of a vector function (the Jacobian) to minimise its sum of squares. When the Jacobian matrix is ill-conditioned, the function varies much faster in some…
We propose a preconditioner that can accelerate the rate of convergence of the Multiple Shooting Shadowing (MSS) method. This recently proposed method can be used to compute derivatives of time-averaged objectives (also known as…
In this paper, we propose a descent method for composite optimization problems with linear operators. Specifically, we first design a structure-exploiting preconditioner tailored to the linear operator so that the resulting preconditioned…
In this paper we propose a new inexact dual decomposition algorithm for solving separable convex optimization problems. This algorithm is a combination of three techniques: dual Lagrangian decomposition, smoothing and excessive gap. The…
Preconditioning has long been a staple technique in optimization, often applied to reduce the condition number of a matrix and speed up the convergence of algorithms. Although there are many popular preconditioning techniques in practice,…
Singular Value Decomposition (SVD) is the basic body of many statistical algorithms and few users question whether SVD is properly handling its job. SVD aims at evaluating the decomposition that best approximates a data matrix, given some…
In this paper, we introduce a powerful technique based on Leave-one-out analysis to the study of low-rank matrix completion problems. Using this technique, we develop a general approach for obtaining fine-grained, entrywise bounds for…
In partial differential equations-based (PDE-based) inverse problems with many measurements, many large-scale discretized PDEs must be solved for each evaluation of the misfit or objective function. In the nonlinear case, evaluating the…
In this work, we discuss two modifications that can be made to a known variational quantum singular value decomposition algorithm popular in the literature. The first is a change to the objective function which hints at improved performance…
Algebraic Riccati equations are encountered in many applications of control and engineering problems, e.g., LQG problems and $H^\infty$ control theory. In this work, we study the properties of one type of discrete-time algebraic Riccati…
Solving a set of simultaneous linear equations is probably the most important topic in numerical methods. For solving linear equations, iterative methods are preferred over the direct methods especially when the coefficient matrix is…
In this paper, we introduce a novel theoretical framework for Gaussian process regression error analysis, leveraging a function-space decomposition. Based on this framework, we develop a weighted Jacobi iterative method that utilizes…