Related papers: A semi-implicit scheme based on Arrow-Hurwicz meth…

In this paper we study the smooth convex-concave saddle point problem. Specifically, we analyze the last iterate convergence properties of the Extragradient (EG) algorithm. It is well known that the ergodic (averaged) iterates of EG…

We consider the minimization of non-convex functions that typically arise in machine learning. Specifically, we focus our attention on a variant of trust region methods known as cubic regularization. This approach is particularly attractive…

We consider a semi-Lagrangian scheme for solving the minimum time problem, with a given target, and the associated eikonal type equation. We first use a discrete time deterministic optimal control problem interpretation of the time…

The splitting method is a powerful method for solving partial differential equations. Various splitting methods have been designed to separate different physics, nonlinearities, and so on. Recently, a new splitting approach has been…

We present a machine learning framework that blends image super-resolution technologies with passive, scalar transport in the level-set method. Here, we investigate whether we can compute on-the-fly, data-driven corrections to minimize…

We study the factor model problem, which aims to uncover low-dimensional structures in high-dimensional datasets. Adopting a robust data-driven approach, we formulate the problem as a saddle-point optimization. Our primary contribution is a…

This paper introduces a smoothed proximal Lagrangian method for minimizing a nonconvex smooth function over a convex domain with additional explicit convex nonlinear constraints. Two key features are 1) the proposed method is single-looped,…

Bilevel optimization problems are receiving increasing attention in machine learning as they provide a natural framework for hyperparameter optimization and meta-learning. A key step to tackle these problems is the efficient computation of…

Loss functions with a large number of saddle points are one of the major obstacles for training modern machine learning models efficiently. First-order methods such as gradient descent are usually the methods of choice for training machine…

In this paper, we propose an inexact perturbed path-following algorithm in the framework of Lagrangian dual decomposition for solving large-scale structured convex optimization problems. Unlike the exact versions considered in literature,…

In this paper, we propose an acceleration framework for a class of iterative methods using the Reduced Order Method (ROM). Assuming that the underlying iterative scheme generates a rich basis for the solution space, we construct the next…

We study randomized sketching methods for approximately solving least-squares problem with a general convex constraint. The quality of a least-squares approximation can be assessed in different ways: either in terms of the value of the…

The systematization of the purely Lagrangean approach to constrained systems in the form of an algorithm involves the iterative construction of a generalized Hessian matrix W taking a rectangular form. This Hessian will exhibit as many left…

We present a novel deep learning approach to approximate the solution of large, sparse, symmetric, positive-definite linear systems of equations. These systems arise from many problems in applied science, e.g., in numerical methods for…

This work is on constrained large-scale non-convex optimization where the constraint set implies a manifold structure. Solving such problems is important in a multitude of fundamental machine learning tasks. Recent advances on Riemannian…

We present a parametric family of semi-implicit second order accurate numerical methods for non-conservative and conservative advection equation for which the numerical solutions can be obtained in a fixed number of forward and backward…

This paper addresses a class of nonsmooth and nonconvex optimization problems defined on complete Riemannian manifolds. The objective function has a composite structure, combining convex, differentiable, and lower semicontinuous terms,…

We propose a practical inexact augmented Lagrangian method (iALM) for nonconvex problems with nonlinear constraints. We characterize the total computational complexity of our method subject to a verifiable geometric condition, which is…

The convergence analysis of a third-order scheme for the highly nonlinear Landau-Lifshitz-Gilbert equation with a non-convex constraint is considered. In this paper, we first present a fully discrete semi-implicit method for solving the…

We consider non-smooth saddle point optimization problems. To solve these problems, we propose a zeroth-order method under bounded or Lipschitz continuous noise, possible adversarial. In contrast to the state-of-the-art algorithms, our…