Related papers: An Iterative Minimization Formulation for Saddle-P…
We propose a constrained high-index saddle dynamics (CHiSD) method to search for index-$k$ saddle points of an energy functional subject to equality constraints. With Riemannian manifold tools, the CHiSD is derived in a minimax framework,…
In this paper, we study the lower iteration complexity bounds for finding the saddle point of a strongly convex and strongly concave saddle point problem: $\min_x\max_yF(x,y)$. We restrict the classes of algorithms in our investigation to…
We propose and analyze several inexact regularized Newton-type methods for finding a global saddle point of convex-concave unconstrained min-max optimization problems. Compared to first-order methods, our understanding of second-order…
We study the iteration complexity of the optimistic gradient descent-ascent (OGDA) method and the extra-gradient (EG) method for finding a saddle point of a convex-concave unconstrained min-max problem. To do so, we first show that both…
The Monge-Kantorovich problem is revisited by means of a variant of the saddle-point method without appealing to $c$-conjugates. A new abstract characterization of the optimal plans is obtained in the case where the cost function takes…
We study a posterior sampling approach to efficient exploration in constrained reinforcement learning. Alternatively to existing algorithms, we propose two simple algorithms that are more efficient statistically, simpler to implement and…
The paper proposes a linesearch for a primal-dual method. Each iteration of the linesearch requires to update only the dual (or primal) variable. For many problems, in particular for regularized least squares, the linesearch does not…
We study the performance of stochastic first-order methods for finding saddle points of convex-concave functions. A notorious challenge faced by such methods is that the gradients can grow arbitrarily large during optimization, which may…
The proliferation of saddle points, rather than poor local minima, is increasingly understood to be a primary obstacle in large-scale non-convex optimization for machine learning. Variable elimination algorithms, like Variable Projection…
This paper introduces a preconditioned method designed to comprehensively address the saddle point system with the aim of improving convergence efficiency. In the preprocessor construction phase, a technical approach for solving the…
We analyze the semi-implicit scheme of high-index saddle dynamics, which provides a powerful numerical method for finding the any-index saddle points and constructing the solution landscape. Compared with the explicit schemes of saddle…
We consider the problem of computing optimal policies in average-reward Markov decision processes. This classical problem can be formulated as a linear program directly amenable to saddle-point optimization methods, albeit with a number of…
We present a scheme improving the minimum-mode following method for finding first order saddle points by confining the displacements of atoms to the subset of those subject to the largest force. By doing so it is ensured that the…
In this paper, a sequential search method for finding the global minimum of an objective function is presented, The descent gradient search is repeated until the global minimum is obtained. The global minimum is located by a process of…
A central challenge to many fields of science and engineering involves minimizing non-convex error functions over continuous, high dimensional spaces. Gradient descent or quasi-Newton methods are almost ubiquitously used to perform such…
A central challenge to many fields of science and engineering involves minimizing non-convex error functions over continuous, high dimensional spaces. Gradient descent or quasi-Newton methods are almost ubiquitously used to perform such…
We propose an approach to saddle point optimization relying only on oracles that solve minimization problems approximately. We analyze its convergence property on a strongly convex--concave problem and show its linear convergence toward the…
Local search heuristics for non-convex optimizations are popular in applied machine learning. However, in general it is hard to guarantee that such algorithms even converge to a local minimum, due to the existence of complicated saddle…
We introduce and investigate stochastic processes designed to find local minimizers and saddle points of non-convex functions, exploring the landscape more efficiently than the standard noisy gradient descent. The processes switch between…
This work presents a numerical analysis of computing transition states of semilinear elliptic partial differential equations (PDEs) via the index-1 saddle dynamics, or equivalently, the gentlest ascent dynamics. To establish clear…