Related papers: Finite-Time Optimization via Scaled Gradient-Momen…
Stochastic optimization algorithms with variance reduction have proven successful for minimizing large finite sums of functions. Unfortunately, these techniques are unable to deal with stochastic perturbations of input data, induced for…
A fast and stable numerical method is formulated to compute the time evolution of a wave function in a magnetic field by solving the time-dependent Schroedinger equation. This computational method is based on the finite element method in…
Large-scale constrained optimization problems are at the core of many tasks in control, signal processing, and machine learning. Notably, problems with functional constraints arise when, beyond a performance{\nobreakdash-}centric goal…
We study the transport properties of nonautonomous chaotic dynamical systems over a finite time duration. We are particularly interested in those regions that remain coherent and relatively non-dispersive over finite periods of time,…
Modern techniques for physical simulations rely on numerical schemes and mesh-refinement methods to address trade-offs between precision and complexity, but these handcrafted solutions are tedious and require high computational power.…
We propose a new explicit pseudo-energy and momentum conserving scheme for the time integration of Hamiltonian systems. The scheme, which is formally second-order accurate, is based on two key ideas: the integration during the time-steps of…
This paper presents a novel stochastic gradient descent algorithm for constrained optimization. The proposed algorithm randomly samples constraints and components of the finite sum objective function and relies on a relaxed logarithmic…
A strongly polynomial algorithm is given for the generalized flow maximization problem. It uses a new variant of the scaling technique, called continuous scaling. The main measure of progress is that within a strongly polynomial number of…
The study of convex optimization has historically been concerned with worst-case convergence rates. The development of the Optimized Gradient Method (OGM), due to \citet{drori2012PerformanceOF,Kim2016optimal}, marked a major milestone in…
The training of modern machine learning models often consists in solving high-dimensional non-convex optimisation problems that are subject to large-scale data. In this context, momentum-based stochastic optimisation algorithms have become…
Two-time-scale optimization is a framework introduced in Zeng et al. (2024) that abstracts a range of policy evaluation and policy optimization problems in reinforcement learning (RL). Akin to bi-level optimization under a particular type…
Computational fluid dynamics (CFD) simulations of viscous fluids described by the Navier-Stokes equations are considered. Depending on the Reynolds number of the flow, the Navier-Stokes equations may exhibit a highly nonlinear behavior. The…
In this paper we propose stochastic gradient-free methods and accelerated methods with momentum for solving stochastic optimization problems. All these methods rely on stochastic directions rather than stochastic gradients. We analyze the…
This paper deals with a network of computing agents aiming to solve an online optimization problem in a distributed fashion, i.e., by means of local computation and communication, without any central coordinator. We propose the gradient…
This paper studies finite-time stability of a class of hybrid systems. We present sufficient conditions in terms of multiple generalized Lyapunov functions for the origin of the hybrid system to be finite-time stable. More specifically, we…
We present a finite-time analysis of two smoothed functional stochastic approximation algorithms for simulation-based optimization. The first is a two time-scale gradient-based method, while the second is a three time-scale Newton-based…
This paper synthesizes anytime algorithms, in the form of continuous-time dynamical systems, to solve monotone variational inequalities. We introduce three algorithms that solve this problem: the projected monotone flow, the safe monotone…
Energy and pollution are urging problems of the 21th century. By gradually changing the actual power grid system, smart grid may evolve into different systems by means of size, elements and strategies, but its fundamental requirements and…
In practice, optimization tasks have some structure that allows developing new algorithms for every problem with faster convergence rates. Using the structure of optimization tasks, we can propose algorithms with more optimistic convergence…
We develop and analyze a highly efficient, second-order time-marching scheme for infinite-dimensional nonlinear geophysical fluid models, designed to accurately approximate invariant measures-that is, the stationary statistical properties…