Related papers: On radially epidifferentiable functions and regula…
Classical analysis of convex and non-convex optimization methods often requires the Lipshitzness of the gradient, which limits the analysis to functions bounded by quadratics. Recent work relaxed this requirement to a non-uniform smoothness…
We study non-convex Hamilton-Jacobi equations in the presence of gradient constraints and produce new, optimal, regularity results for the solutions. A distinctive feature of those equations regards the existence of a lower bound to the…
We consider nonlinear multistage stochastic optimization problems in the spaces of integrable functions. We allow for nonlinear dynamics and general objective functionals, including dynamic risk measures. We study causal operators…
The primary goal of the paper is to establish characteristic properties of (extended) real-valued functions defined on normed vector spaces that admit the representation as the lower envelope of their minimal (with respect to pointwise…
Continuous optimization is an important problem in many areas of AI, including vision, robotics, probabilistic inference, and machine learning. Unfortunately, most real-world optimization problems are nonconvex, causing standard convex…
This paper presents a framework of successive functional gradient optimization for training nonconvex models such as neural networks, where training is driven by mirror descent in a function space. We provide a theoretical analysis and…
Exhausters are families of convex compact sets that allow one to represent directional derivative of the studied function at the considered point in the form of InfMax or SupMin of linear functions. Functions for which such a representation…
We propose a family of optimization methods that achieve linear convergence using first-order gradient information and constant step sizes on a class of convex functions much larger than the smooth and strongly convex ones. This larger…
We provide a unified framework for a systematic analysis of the existence of solutions to general nonconvex problems, relying on asymptotic and retractive cones for functions and sets. Using this framework we develop new necessary and…
The nonlinear conjugate gradient methods are known to be an effective approach for standard unconstrained optimization problems especially for large-scale problems. This paper proposes a proximal nonlinear conjugate gradient method, which…
The purpose of the paper is to give a complete characterization of the continuity of lower envelopes in the infinite dimensional spaces in terms of the notion of c-regularity. As an application we introduce a variational unconstrained…
We investigate a class of composite nonconvex functions, where the outer function is the sum of univariate extended-real-valued convex functions and the inner function is the limit of difference-of-convex functions. A notable feature of…
We consider the proximal gradient method on Riemannian manifolds for functions that are possibly not geodesically convex. Starting from the forward-backward-splitting, we define an intrinsic variant of the proximal gradient method that uses…
As a starting point of our research, we show that, for a fixed order $\gamma\geq 1$, each local minimizer of a rather general nonsmooth optimization problem in Euclidean spaces is either M-stationary in the classical sense (corresponding to…
This paper considers the analysis of continuous time gradient-based optimization algorithms through the lens of nonlinear contraction theory. It demonstrates that in the case of a time-invariant objective, most elementary results on…
Variational convexity, together with ist strong counterpart, of extended-real-valued functions has been recently introduced by Rockafellar. In this paper we present second-order characterizations of these properties, i.e., conditions using…
In this work, we consider constrained stochastic optimization problems under hidden convexity, i.e., those that admit a convex reformulation via non-linear (but invertible) map $c(\cdot)$. A number of non-convex problems ranging from…
In this work we focus our attention on distributed optimization problems in the context where the communication time between the server and the workers is non-negligible. We obtain novel methods supporting bidirectional compression (both…
Non-convex optimization is ubiquitous in modern machine learning. Researchers devise non-convex objective functions and optimize them using off-the-shelf optimizers such as stochastic gradient descent and its variants, which leverage the…
We generalize and improve the original characterization given by Valadier [20, Theorem 1] of the subdifferential of the pointwise supremum of convex functions, involving the subdifferentials of the data functions at nearby points. We remove…