Related papers: Symbol based convergence analysis in multigrid met…
Saddle-point problems have recently gained increased attention from the machine learning community, mainly due to applications in training Generative Adversarial Networks using stochastic gradients. At the same time, in some applications…
In recent years, there has been a renewed interest in preconditioning for multilevel Toeplitz systems, a research field that has been extensively explored over the past several decades. This work introduces novel preconditioning strategies…
Primal-dual methods for solving convex optimization problems with functional constraints often exhibit a distinct two-stage behavior. Initially, they converge towards a solution at a sublinear rate. Then, after a certain point, the method…
We consider strongly-convex-strongly-concave saddle-point problems with general non-bilinear objective and different condition numbers with respect to the primal and the dual variables. First, we consider such problems with smooth composite…
We develop eigenvalue bounds for symmetric, block tridiagonal multiple saddle-point linear systems, preconditioned with block diagonal matrices. We extend known results for $3 \times 3$ block systems [Bradley and Greif, IMA J.\ Numer. Anal.…
A central challenge to using first-order methods for optimizing nonconvex problems is the presence of saddle points. First-order methods often get stuck at saddle points, greatly deteriorating their performance. Typically, to escape from…
We consider symmetric positive definite preconditioners for multiple saddle-point systems of block tridiagonal form, which can be applied within the MINRES algorithm. We describe such a preconditioner for which the preconditioned matrix has…
This paper introduces a preconditioned method designed to comprehensively address the saddle point system with the aim of improving convergence efficiency. In the preprocessor construction phase, a technical approach for solving the…
In this paper, we present novel randomized algorithms for solving saddle point problems whose dual feasible region is given by the direct product of many convex sets. Our algorithms can achieve an ${\cal O}(1/N)$ and ${\cal O}(1/N^2)$ rate…
We construct multigrid methods for an elliptic distributed optimal control problem that are robust with respect to a regularization parameter. We prove the uniform convergence of the $W$-cycle algorithm and demonstrate the performance of…
A modification of the generalized shift-splitting (GSS) method is presented for solving singular saddle point problems. In this kind of modification, the diagonal shift matrix is replaced by a block diagonal matrix which is symmetric…
We consider the problem of convergence to a saddle point of a concave-convex function via gradient dynamics. Since first introduced by Arrow, Hurwicz and Uzawa in [1] such dynamics have been extensively used in diverse areas, there are,…
In this article, we study the convergence of Mirror Descent (MD) and Optimistic Mirror Descent (OMD) for saddle point problems satisfying the notion of coherence as proposed in Mertikopoulos et al. We prove convergence of OMD with exact…
The paper studies a distributed gradient descent (DGD) process and considers the problem of showing that in nonconvex optimization problems, DGD typically converges to local minima rather than saddle points. The paper considers…
We derive eigenvalue bounds for symmetric block-tridiagonal multiple saddle-point systems preconditioned with block-diagonal Schur complement matrices. This analysis applies to an arbitrary number of blocks and accounts for the case where…
We consider the generalized successive overrelaxation (GSOR) method for solving a class of block three-by-three saddle-point problems. Based on the necessary and sufficient conditions for all roots of a real cubic polynomial to have modulus…
We address the problem of preconditioning a sequence of saddle point linear systems arising in the solution of PDE-constrained optimal control problems via active-set Newton methods, with control and (regularized) state constraints. We…
The primal-dual hybrid gradient (PDHG) method is one of the most popular algorithms for solving saddle point problems. However, when applying the PDHG method and its many variants to some real-world models commonly encountered in signal…
We provide an overview of primal-dual algorithms for nonsmooth and non-convex-concave saddle-point problems. This flows around a new analysis of such methods, using Bregman divergences to formulate simplified conditions for convergence.
We present a parallelized primal-dual algorithm for solving constrained convex optimization problems. The algorithm is "block-based," in that vectors of primal and dual variables are partitioned into blocks, each of which is updated only by…