Related papers: Classifying extrema using intervals

We prove that the smallest minimizer s(f) of a real convex function f is less than or equal to a real point x if and only if the right derivative of f at x is non-negative. Similarly, the largest minimizer t(f) is greater or equal to x if…

We prove the exact worst-case convergence rate of gradient descent for smooth strongly convex optimization, with respect to the performance criterion $\Vert \nabla f(x_N)\Vert^2/(f(x_0)-f_*)$. The proof differs from the previous one by…

Estimation of linear functionals from observed data is an important task in many subjects. Juditsky & Nemirovski [The Annals of Statistics 37.5A (2009): 2278-2300] propose a framework for non-parametric estimation of linear functionals in a…

The min-max problem, also known as the saddle point problem, is a class of optimization problems which minimizes and maximizes two subsets of variables simultaneously. This class of problems can be used to formulate a wide range of signal…

In this paper, we propose an unconstrained framework for eigenvalue problems in both discrete and continuous settings. We begin our discussion to solve a generalized eigenvalue problem $A{\bf x} = \lambda B{\bf x}$ with two $N\times N$ real…

Machine learning algorithms typically perform optimization over a class of non-convex functions. In this work, we provide bounds on the fundamental hardness of identifying the global minimizer of a non convex function. Specifically, we…

We present methods that provide all zeroes and extrema of a function that do not require differentiation. Using point process theory, we are able to describe the locations of zeroes or maxima, their number, as well as their distribution…

In this paper we investigate the convergence of a recently popular class of first-order primal-dual algorithms for saddle point problems under the presence of errors occurring in the proximal maps and gradients. We study several types of…

The proximal point algorithm, which is a well-known tool for finding minima of convex functions, is generalized from the classical Hilbert space framework into a nonlinear setting, namely, geodesic metric spaces of nonpositive curvature. We…

We study the performance of stochastic first-order methods for finding saddle points of convex-concave functions. A notorious challenge faced by such methods is that the gradients can grow arbitrarily large during optimization, which may…

Beck and Teboulle's FISTA method for finding a minimizer of the sum of two convex functions, one of which has a Lipschitz continuous gradient whereas the other may be nonsmooth, is arguably the most important optimization algorithm of the…

We study the factor model problem, which aims to uncover low-dimensional structures in high-dimensional datasets. Adopting a robust data-driven approach, we formulate the problem as a saddle-point optimization. Our primary contribution is a…

We propose and analyze asymptotic proximal point (APP) methods to find the global minimizer for a class of nonconvex, nonsmooth, or even discontinuous multiple minima functions. The method is based on an asymptotic representation of…

The proliferation of saddle points, rather than poor local minima, is increasingly understood to be a primary obstacle in large-scale non-convex optimization for machine learning. Variable elimination algorithms, like Variable Projection…

We develop a variational minimax method for detecting maximal saddle-node bifurcations in abstract nonlinear equations. Unlike continuation and path-following techniques, the method identifies the critical parameter directly as an extremal…

Local search heuristics for non-convex optimizations are popular in applied machine learning. However, in general it is hard to guarantee that such algorithms even converge to a local minimum, due to the existence of complicated saddle…

We establish sharp well-posedness and approximation estimates for variational saddle point systems at the continuous level. The main results of this note have been known to be true only in the finite dimensional case. Known spectral results…

This paper studies properties of fixed points of generalised Extra-gradient (GEG) algorithms applied to min-max problems. We discuss connections between saddle points of the objective function of the min-max problem and GEG fixed points. We…

Given a discrete function $f:\Z^d \to \R$ we consider the maximal operator $$Mf(\vec{n}) = \sup_{r\geq0} \frac{1}{N(r)} \sum_{\vec{m} \in \bar{\Omega}_r} \big|f(\vec{n} + \vec{m})\big|,$$ where $\big\{\bar{\Omega}_r\big\}_{r \geq 0}$ are…

Standard approaches to stochastic gradient estimation, with only noisy black-box function evaluations, use the finite-difference method or its variants. While natural, it is open to our knowledge whether their statistical accuracy is the…