Related papers: Newton-like dynamics associated to nonconvex optim…
Machine learning problems such as neural network training, tensor decomposition, and matrix factorization, require local minimization of a nonconvex function. This local minimization is challenged by the presence of saddle points, of which…
In this paper, we propose objective-function-free (OFF) variants of the proximal Newton method for nonconvex composite optimization problems and the regularized Newton method for unconstrained optimization problems, respectively, using…
Motivated by variational models in continuum mechanics, we introduce a novel algorithm to perform nonsmooth and nonconvex minimizations with linear constraints in Euclidean spaces. We show how this algorithm is actually a natural…
We extend the standard notion of self-concordance to non-convex optimization and develop a family of second-order algorithms with global convergence guarantees. In particular, two function classes -- \textit{weakly self-concordant}…
The problem of constant-speed ballistics is studied under the umbrella of non-linear non-holonomic constrained systems. The Newtonian approach is shown to be equivalent to the use of Chetaev's rule to incorporate the constraint within the…
A level orbit of a mechanical Hamiltonian system is a solution of Newton equation that is contained in a level set of the potential energy. In 2003, Mark Levi asked for a characterization of the smooth potential energy functions on the…
We propose a derivative-free trust-region method based on finite-difference gradient approximations for smooth optimization problems with convex constraints. The proposed method does not require computing an approximate stationarity…
This paper deals with local criteria for the convergence to a global minimiser for gradient flow trajectories and their discretisations. To obtain quantitative estimates on the speed of convergence, we consider variations on the classical…
It is known that when minimizing smooth Polyak-{\L}ojasiewicz (PL) functions, momentum algorithms cannot significantly improve the convergence bound of gradient descent, contrasting with the acceleration phenomenon occurring in the strongly…
It has been recently discovered that a convex function can be determined by its slopes and its infimum value, provided this latter is finite. The result was extended to nonconvex functions by replacing the infimum value by the set of all…
In this letter we study the proximal gradient dynamics. This recently-proposed continuous-time dynamics solves optimization problems whose cost functions are separable into a nonsmooth convex and a smooth component. First, we show that the…
We study the asymptotic behavior of the trajectory of a nonautonomous evolution equation governed by a quasi-nonexpansive operator in Hilbert spaces. We prove the weak convergence of the trajectory to a fixed point of the operator by…
We uncover the gradient structure to investigate the convergence of solutions in nonlocal nonlinear dynamical systems. Mainly but not exclusively, we use the Lojasiewicz inequality to prove convergence results in various spaces with…
The asymptotic analysis of a generic stochastic optimization algorithm mainly relies on the establishment of a specific descent condition. While the convexity assumption allows for technical shortcuts and generally leads to strict…
First order optimization algorithms play a major role in large scale machine learning. A new class of methods, called adaptive algorithms, were recently introduced to adjust iteratively the learning rate for each coordinate. Despite great…
We consider stochastic optimization problems involving an expected value of a nonlinear function of a base random vector and a conditional expectation of another function depending on the base random vector, a dependent random vector, and…
We begin by considering second order dynamical systems of the from $\ddot x(t) + \gamma(t)\dot x(t) + \lambda(t)B(x(t))=0$, where $B: {\cal H}\rightarrow{\cal H}$ is a cocoercive operator defined on a real Hilbert space ${\cal H}$,…
We state and prove a stabilisation result for solutions of abstract gradient systems associated with nonsmooth energy functions on infinite dimensional Hilbert spaces. One feature is that in this general setting the assumption on the range…
The non-monotonic propagation of fronts is considered. When the speed function $F:\mathbb{R}^{n} \times [0,T]\rightarrow \mathbb{R}$ is prescribed, the non-linear advection equation $\phi_{t}+F|\nabla \phi|=0$ is a Hamilton-Jacobi equation…
We study the behavior of the trajectories of a second-order differential equation with vanishing damping, governed by the Yosida regularization of a maximally monotone operator with time-varying index, along with a new {\em Regularized…