Related papers: Accelerated methods for composite non-bilinear sad…
Saddle point problems arise in a variety of applications, e.g., when solving the Stokes equations. They can be formulated such that the system matrix is symmetric, but indefinite, so the variational convergence theory that is usually used…
In this paper we address the speed planning problem for a vehicle along a predefined path. A weighted average of two (conflicting) terms, energy consumption and travel time, is minimized. After deriving a non-convex mathematical model of…
We analyze stochastic gradient descent for optimizing non-convex functions. In many cases for non-convex functions the goal is to find a reasonable local minimum, and the main concern is that gradient updates are trapped in saddle points.…
We consider a generic convex-concave saddle point problem with separable structure, a form that covers a wide-ranged machine learning applications. Under this problem structure, we follow the framework of primal-dual updates for saddle…
Bilevel optimization has been developed for many machine learning tasks with large-scale and high-dimensional data. This paper considers a constrained bilevel optimization problem, where the lower-level optimization problem is convex with…
In this paper, we deal with multiobjective composite optimization problems, where each objective function is a combination of smooth and possibly non-smooth functions. We first propose a parameter-dependent conditional gradient method to…
In this article we investigate the possibilities of accelerating the double smoothing technique when solving unconstrained nondifferentiable convex optimization problems. This approach relies on the regularization in two steps of the…
We consider the problem of convergence to a saddle point of a concave-convex function via gradient dynamics. Since first introduced by Arrow, Hurwicz and Uzawa in [1] such dynamics have been extensively used in diverse areas, there are,…
In this paper, we extend the inexact Uzawa algorithm in [Q. Hu, J. Zou, SIAM J. Matrix Anal., 23(2001), pp. 317-338] to the nonsymmetric generalized saddle point problem. The techniques used here are similar to those in [Bramble \emph{et…
Large-scale saddle-point problems arise in such machine learning tasks as GANs and linear models with affine constraints. In this paper, we study distributed saddle-point problems (SPP) with strongly-convex-strongly-concave smooth…
We study the \emph{Proximal Alternating Predictor-Corrector} (PAPC) algorithm introduced recently by Drori, Sabach and Teboulle to solve nonsmooth structured convex-concave saddle point problems consisting of the sum of a smooth convex…
This paper introduces new parameter-free first-order methods for convex optimization problems in which the objective function exhibits H\"{o}lder smoothness. Inspired by the recently proposed distance-over-gradient (DOG) technique, we…
In this paper, we propose a preconditioner based on the shift-splitting method for generalized saddle point problems with nonsymmetric positive definite (1,1)-block and symmetric positive semidefinite $(2,2)$-block. The proposed…
A class of monotone operator equations, which can be decomposed into sum of the gradient of a strongly convex function and a linear and skew-symmetric operator, is considered in this work. Based on discretization of the generalized gradient…
In this paper, we present novel randomized algorithms for solving saddle point problems whose dual feasible region is given by the direct product of many convex sets. Our algorithms can achieve an ${\cal O}(1/N)$ and ${\cal O}(1/N^2)$ rate…
The paper studies a distributed gradient descent (DGD) process and considers the problem of showing that in nonconvex optimization problems, DGD typically converges to local minima rather than saddle points. The paper considers…
We investigate degenerate saddle point problems, which can be viewed as limit cases of standard mixed formulations of symmetric problems with large jumps in coefficients. We prove that they are well-posed in a standard norm despite the…
The dual formulation for linear elasticity, in contrast to the primal formulation, is not affected by locking, as it is based on the stresses as main unknowns. Thus it is quite attractive for nearly incompressible and incompressible…
Inspired by the Optimistic Gradient Ascent-Proximal Point Algorithm (OGAProx) proposed by Bo{\c{t}}, Csetnek, and Sedlmayer for solving a saddle-point problem associated with a convex-concave function with a nonsmooth coupling function and…
We analyze stochastic algorithms for optimizing nonconvex, nonsmooth finite-sum problems, where the nonconvex part is smooth and the nonsmooth part is convex. Surprisingly, unlike the smooth case, our knowledge of this fundamental problem…