Related papers: Towards faster first order methods: A continuous-t…

This work proposes an accelerated first-order algorithm we call the Robust Momentum Method for optimizing smooth strongly convex functions. The algorithm has a single scalar parameter that can be tuned to trade off robustness to gradient…

We study nonsmooth convex minimization through a continuous-time dynamical system that can be seen as a high-resolution ODE of Nesterov Accelerated Gradient (NAG) adapted to the nonsmooth case. We apply a time-varying Moreau envelope…

We develop and implement a novel fast bootstrap for dependent data. Our scheme is based on the i.i.d. resampling of the smoothed moment indicators. We characterize the class of parametric and semi-parametric estimation problems for which…

We present the first accelerated randomized algorithm for solving linear systems in Euclidean spaces. One essential problem of this type is the matrix inversion problem. In particular, our algorithm can be specialized to invert positive…

This Ph.D. dissertation contains results in two different but related fields: the implementation of model predictive control (MPC) in embedded systems using first order methods, and restart schemes for accelerated first order methods…

Diffusion models generate high-quality synthetic data. They operate by defining a continuous-time forward process which gradually adds Gaussian noise to data until fully corrupted. The corresponding reverse process progressively "denoises"…

In order to minimize a differentiable geodesically convex function, we study a second-order dynamical system on Riemannian manifolds with an asymptotically vanishing damping term of the form $\alpha/t$. For positive values of $\alpha$,…

Treating optimization methods as dynamical systems can be traced back centuries ago in order to comprehend the notions and behaviors of optimization methods. Lately, this mind set has become the driving force to design new optimization…

We present an inference scheme of long timescale, non-exponential kinetics from Molecular Dynamics simulations accelerated by stochastic resetting. Standard simulations provide valuable insight into chemical processes but are limited to…

We present a conservative/dissipative time integration scheme for nonlinear mechanical systems. Starting from a weak form, we derive algorithmic forces and velocities that guarantee the desired conservation/dissipation properties. Our…

First-order operator splitting methods are ubiquitous among many fields through science and engineering, such as inverse problems, signal/image processing, statistics, data science and machine learning, to name a few. In this paper, we…

In this paper we study several classes of stochastic optimization algorithms enriched with heavy ball momentum. Among the methods studied are: stochastic gradient descent, stochastic Newton, stochastic proximal point and stochastic dual…

The majority of machine learning methods can be regarded as the minimization of an unavailable risk function. To optimize the latter, given samples provided in a streaming fashion, we define a general stochastic Newton algorithm and its…

It is known that the restarted full orthogonalization method (FOM) outperforms the restarted generalized minimum residual (GMRES) method in several circumstances for solving shifted linear systems when the shifts are handled simultaneously.…

Restart -- interrupting a stochastic process followed by a new start -- is known to improve the mean time to its completion, and the general conditions under which such an improvement is achieved are now well understood. Here, we explore…

A classical reduced order model for dynamical problems involves spatial reduction of the problem size. However, temporal reduction accompanied by the spatial reduction can further reduce the problem size without losing accuracy much, which…

In a Hilbert space, we propose a class of general mixed-order primal-dual dynamical systems with Tikhonov regularization for a convex optimization problem with linear equality constraints. The proposed dynamical system is characterized by…

Bilevel optimization has arisen as a powerful tool in modern machine learning. However, due to the nested structure of bilevel optimization, even gradient-based methods require second-order derivative approximations via Jacobian- or/and…

We investigate the (functional) convex order of for various continuous martingale processes, either with respect to their diffusions coefficients for L\'evy-driven SDEs or their integrands for stochastic integrals. Main results are bordered…

First-order stochastic methods are the state-of-the-art in large-scale machine learning optimization owing to efficient per-iteration complexity. Second-order methods, while able to provide faster convergence, have been much less explored…