Related papers: Coordinate-wise descent methods for leading eigenv…
A new approach is discussed for solving large nonsymmetric systems of linear equations with multiple right-hand sides. The first system is solved with a deflated GMRES method that generates eigenvector information at the same time that the…
Non-convex optimization problems are challenging to solve; the success and computational expense of a gradient descent algorithm or variant depend heavily on the initialization strategy. Often, either random initialization is used or…
We focus on solving constrained convex optimization problems using mini-batch stochastic gradient descent. Dynamic sample size rules are presented which ensure a descent direction with high probability. Empirical results from two…
Convex separable quadratic optimization problems occur in many practical applications. In this paper, based on an iterative resolution scheme of the KKT system, we develop an efficient method for solving a quadratic programming problem with…
The history of research on eigenvalue problems is rich with many outstanding contributions. Nonetheless, the rapidly increasing size of data sets requires new algorithms for old problems in the context of extremely large matrix dimensions.…
Stochastic optimization methods have been hugely successful in making large-scale optimization problems feasible when computing the full gradient is computationally prohibitive. Using the theory of modified equations for numerical…
We present a novel randomized block coordinate descent method for the minimization of a convex composite objective function. The method uses (approximate) partial second-order (curvature) information, so that the algorithm performance is…
A deflated restarted Lanczos algorithm is given for both solving symmetric linear equations and computing eigenvalues and eigenvectors. The restarting limits the storage so that finding eigenvectors is practical. Meanwhile, the deflating…
A new approach to solving eigenvalue optimization problems for large structured matrices is proposed and studied. The class of optimization problems considered is related to computing structured pseudospectra and their extremal points, and…
Inspired by multigrid methods for linear systems of equations, multilevel optimization methods have been proposed to solve structured optimization problems. Multilevel methods make more assumptions regarding the structure of the…
The vertical modes of linearized equations of motion are widely used by the oceanographic community in numerous theoretical and observational contexts. However, the standard approach for solving the generalized eigenvalue problem using…
In this paper, we proposed a monotone block coordinate descent method for solving absolute value equation (AVE). Under appropriate conditions, we analyzed the global convergence of the algorithm and conduct numerical experiments to…
In this paper we present a convergence rate analysis of inexact variants of several randomized iterative methods. Among the methods studied are: stochastic gradient descent, stochastic Newton, stochastic proximal point and stochastic…
With the development of machine learning and Big Data, the concepts of linear and non-linear optimization techniques are becoming increasingly valuable for many quantitative disciplines. Problems of that nature are typically solved using…
In this work, the author presents a novel method for finding descent directions shared by two or more differentiable functions defined on the same unconstrained domain space. Then, the author illustrates an alternative Multiple-Gradient…
We present a novel, practical, and provable approach for solving diagonally constrained semi-definite programming (SDP) problems at scale using accelerated non-convex programming. Our algorithm non-trivially combines acceleration motions…
A common challenge faced in quantum physics is finding the extremal eigenvalues and eigenvectors of a Hamiltonian matrix in a vector space so large that linear algebra operations on general vectors are not possible. There are numerous…
This paper introduces an efficient algorithm for finding the dominant generalized eigenvectors of a pair of symmetric matrices. Combining tools from approximation theory and convex optimization, we develop a simple scalable algorithm with…
Constrained non-convex optimization problems frequently arise in control applications. Solving such problems is inherently challenging, as existing methods often converge to suboptimal local minima or incur prohibitive computational costs.…
Versions of GMRES with deflation of eigenvalues are applied to lattice QCD problems. Approximate eigenvectors corresponding to the smallest eigenvalues are generated at the same time that linear equations are solved. The eigenvectors…