Related papers: Two-Level Sketching Alternating Anderson accelerat…
The Alternating Direction Method of Multipliers (ADMM) has gained significant attention across a broad spectrum of machine learning applications. Incorporating the over-relaxation technique shows potential for enhancing the convergence rate…
One of the most computationally expensive steps of the low-rank ADI method for large-scale Lyapunov equations is the solution of a shifted linear system at each iteration. We propose the use of the extended Krylov subspace method for this…
This paper studies the performance Newton's iteration applied with Anderson acceleration for solving the incompressible steady Navier-Stokes equations. We manifest that this method converges superlinearly with a good initial guess, and…
We consider two modifications of the Arrow-Hurwicz (AH) iteration for solving the incompressible steady Navier-Stokes equations for the purpose of accelerating the algorithm: grad-div stabilization, and Anderson acceleration. AH is a…
The problem of finding a vector with the fewest nonzero elements that satisfies an underdetermined system of linear equations is an NP-complete problem that is typically solved numerically via convex heuristics or nicely-behaved nonconvex…
We propose a new randomized algorithm for solving L2-regularized least-squares problems based on sketching. We consider two of the most popular random embeddings, namely, Gaussian embeddings and the Subsampled Randomized Hadamard Transform…
In this paper, we propose a low-rank coordinate descent approach to structured semidefinite programming with diagonal constraints. The approach, which we call the Mixing method, is extremely simple to implement, has no free parameters, and…
In this paper we continue our work on adaptive timestep control for weakly non- stationary problems. The core of the method is a space-time splitting of adjoint error representations for target functionals due to S\"uli and Hartmann. The…
In many data structure settings, it has been shown that using "double hashing" in place of standard hashing, by which we mean choosing multiple hash values according to an arithmetic progression instead of choosing each hash value…
In this paper, we study the convergence of Alternating Projection (AP) algorithm for the matrix completion and compressed sensing problems. We also present computational evidence for the excellent performance of the algorithm. Also, in the…
Stochastic bilevel optimization, which captures the inherent nested structure of machine learning problems, is gaining popularity in many recent applications. Existing works on bilevel optimization mostly consider either unconstrained…
The alternating direction method of multipliers (ADMM) has found widespread use in solving separable convex optimization problems. In this paper, by employing Nesterov extrapolation technique, we propose two families of accelerated…
In this paper we introduce a novel abstract descent scheme suited for the minimization of proper and lower semicontinuous functions. The proposed abstract scheme generalizes a set of properties that are crucial for the convergence of…
The variational approach to fracture is effective for simulating the nucleation and propagation of complex crack patterns, but is computationally demanding. The model is a strongly nonlinear non-convex variational inequality that demands…
The alternating gradient descent (AGD) is a simple but popular algorithm which has been applied to problems in optimization, machine learning, data ming, and signal processing, etc. The algorithm updates two blocks of variables in an…
Maximum a posteriori (MAP) inference over discrete Markov random fields is a fundamental task spanning a wide spectrum of real-world applications, which is known to be NP-hard for general graphs. In this paper, we propose a novel…
Nonconvex and nonsmooth problems have recently attracted considerable attention in machine learning. However, developing efficient methods for the nonconvex and nonsmooth optimization problems with certain performance guarantee remains a…
Stochastic nested optimization, including stochastic compositional, min-max and bilevel optimization, is gaining popularity in many machine learning applications. While the three problems share the nested structure, existing works often…
The Alternating Minimization Algorithm (AMA) has been proposed by Tseng to solve convex programming problems with two-block separable linear constraints and objectives, whereby (at least) one of the components of the latter is assumed to be…
In this paper, we introduce two novel parallel projection methods for finding a solution of a system of variational inequalities which is also a common fixed point of a family of (asymptotically) $\kappa$ - strict pseudocontractive…