Related papers: Been Down So Long it Looks Like Up to Me: A Unifie…
We propose a mini-batching scheme for improving the theoretical complexity and practical performance of semi-stochastic gradient descent applied to the problem of minimizing a strongly convex composite function represented as the sum of an…
In this paper we consider a composite optimization problem that minimizes the sum of a weakly smooth function and a convex function with either a bounded domain or a uniformly convex structure. In particular, we first present a…
This article proposes new perspectives for developing derivative based numerical algorithms, supported by the introduction of a generalized derivative operators. It demonstrates that these operators have the potential to enhance and extend…
Computing explicitly the {\epsilon}-subdifferential of a proper function amounts to computing the level set of a convex function namely the conjugate minus a linear function. The resulting theoretical algorithm is applied to the the class…
In this article, we describe a function fitting method that has potential applications in machine learning and also prove relevant theorems. The described function fitting method is a convex minimization problem and can be solved using a…
In this paper, we study the convergence of an interior subgradient and proximal methods for a DC (difference of convex functions) constrained minimization problem.
In this work, we investigate stochastic quasi-Newton methods for minimizing a finite sum of cost functions over a decentralized network. In Part I, we develop a general algorithmic framework that incorporates stochastic quasi-Newton…
The motivation for this paper stems from the desire to develop an adaptive sampling method for solving constrained optimization problems in which the objective function is stochastic and the constraints are deterministic. The method…
Drawdown (resp. drawup) of a stochastic process, also referred as the reflected process at its supremum (resp. infimum), has wide applications in many areas including financial risk management, actuarial mathematics and statistics. In this…
In this paper, we study the performance of a large family of SGD variants in the smooth nonconvex regime. To this end, we propose a generic and flexible assumption capable of accurate modeling of the second moment of the stochastic…
We present a new feasible proximal gradient method for constrained optimization where both the objective and constraint functions are given by the summation of a smooth, possibly nonconvex function and a convex simple function. The…
We introduce a variational approach for extracting curves between a list of possible endpoints, based on the discretization of an energy and Smirnov's decomposition theorem for vector fields. It is used to design a bi-level minimization…
Stochastic Gradient Descent-Ascent (SGDA) is one of the most prominent algorithms for solving min-max optimization and variational inequalities problems (VIP) appearing in various machine learning tasks. The success of the method led to…
We consider the problem of minimizing a function, which is the sum of a linear function and a composition of a strongly convex function with a linear transformation, over a compact polyhedral set. Jaggi and Lacoste-Julien [14] showed that…
This paper proposes a dynamical Variable-separation method for solving parameter-dependent dynamical systems. To achieve this, we establish a dynamical low-rank approximation for the solutions of these dynamical systems by successively…
In this paper, we present a generic framework to extend existing uniformly optimal convex programming algorithms to solve more general nonlinear, possibly nonconvex, optimization problems. The basic idea is to incorporate a local search…
Level proximal subdifferential was introduced by Rockafellar recently for studying proximal mappings of possibly nonconvex functions. In this paper a systematic study of level proximal subdifferential is given. We characterize variational…
This paper presents an algorithm for solving multiobjective optimization problems involving composite functions, where we minimize a quadratic model that approximates $F(x) - F(x^k)$ and that can be derivative-free. We establish theoretical…
The gradient method for minimize a differentiable convex function on Riemannian manifolds with lower bounded sectional curvature is analyzed in this paper. The analysis of the method is presented with three different finite procedures for…
Approximation of subdifferentials is one of the main tasks when computing descent directions for nonsmooth optimization problems. In this article, we propose a bisection method for weakly lower semismooth functions which is able to compute…