Related papers: Steepest geometric descent for regularized quasico…

The starting assumptions to study the convergence and complexity of gradient-type methods may be the smoothness (also called Lipschitz continuity of gradient) and the strong convexity. In this note, we revisit these two basic properties…

In this paper we consider the steepest descent L2-gradient flow of the entropy functional. The flow expands convex curves, with the radius of an initial circle growing like the square root of time. Our main result is that, for any initial…

The present article studies the minimization of convex, L-smooth functions defined on a separable real Hilbert space. We analyze regularized stochastic gradient descent (reg-SGD), a variant of stochastic gradient descent that uses a…

Stochastic gradient descent is one of the most successful approaches for solving large-scale problems, especially in machine learning and statistics. At each iteration, it employs an unbiased estimator of the full gradient computed from one…

We consider the minimization of an M-convex function, which is a discrete convexity concept for functions on the integer lattice points. It is known that a minimizer of an Mconvex function can be obtained by the steepest descent algorithm.…

Non-convex optimization is ubiquitous in modern machine learning. Researchers devise non-convex objective functions and optimize them using off-the-shelf optimizers such as stochastic gradient descent and its variants, which leverage the…

We establish local regularity results for minimizers of autonomous vectorial integrals of Calculus of Variations, assuming $\psi$-growth conditions and imposing $\varphi$-quasiconvexity only in an asymptotic sense, both in the sub-quadratic…

Typically, the sequence of points generated by an optimization algorithm may have multiple limit points. Under convexity assumptions, however, (sub)gradient methods are known to generate a convergent sequence of points. In this paper, we…

The (delta-) normal cone to an arbitrary intersection of sublevel sets of proper, lower semicontinuous, and convex functions is characterized, using either epsilon-subdifferentials at the nominal point or exact subdifferentials at nearby…

Estimation of convex functions finds broad applications in engineering and science, while convex shape constraint gives rise to numerous challenges in asymptotic performance analysis. This paper is devoted to minimax optimal estimation of…

We adapt the quasi-monotone method from [2] for composite convex minimization in the stochastic setting. For the proposed numerical scheme we derive the optimal convergence rate in terms of the last iterate, rather than on average as it is…

This paper deals with convex nonsmooth optimization problems. We introduce a general smooth approximation framework for the original function and apply random (accelerated) coordinate descent methods for minimizing the corresponding smooth…

Composite minimization involves a collection of functions which are aggregated in a nonsmooth manner. It covers, as a particular case, smooth approximation of minimax games, minimization of max-type functions, and simple composite…

Optimization problems, arise in many practical applications, from the view points of both theory and numerical methods. Especially, significant improvement in deep learning training came from the Quasi-Newton methods. Quasi-Newton search…

We establish partial regularity results for minimizers of a class of functionals depending on differential expressions based on elliptic operators. Specifically, we focus on functionals of Orlicz growth with a natural strong quasiconvexity…

We show a global existence result of weak solutions for a class of generalized Surface Quasi-Geostrophic equation in the inviscid case. We also prove the global regularity of such solutions for the equation with slightly supercritical…

This paper studies a class of simple bilevel optimization problems where we minimize a composite convex function at the upper-level subject to a composite convex lower-level problem. Existing methods either provide asymptotic guarantees for…

Constrained quasiconvex optimization problems appear in many fields, such as economics, engineering, and management science. In particular, fractional programming, which models ratio indicators such as the profit/cost ratio as fractional…

The paper is devoted to new modifications of recently proposed adaptive methods of Mirror Descent for convex minimization problems in the case of several convex functional constraints. Methods for problems of two classes are considered. The…

In this paper, we study when we might expect the optimization curve induced by gradient descent to be \emph{convex} -- precluding, for example, an initial plateau followed by a sharp decrease, making it difficult to decide when optimization…