Related papers: Optimization with Momentum: Dynamical, Control-The…
In this paper, we generalize the chance optimization problems and introduce constrained volume optimization where enables us to obtain convex formulation for challenging problems in systems and control. We show that many different problems…
We study the optimal control of discrete time mean filed dynamical systems under partial observations. We express the global law of the filtered process as a controlled system with its own dynamics. Following a dynamic programming approach,…
Decentralized stochastic optimization has emerged as a fundamental paradigm for large-scale machine learning. However, practical implementations often rely on biased gradient estimators arising from communication compression or inexact…
Stochastic Gradient Descent (SGD) methods see many uses in optimization problems. Modifications to the algorithm, such as momentum-based SGD methods have been known to produce better results in certain cases. Much of this, however, is due…
We consider a class of combinatorial optimization problems that emerge in a variety of domains among which: condensed matter physics, theory of financial risks, error correcting codes in information transmissions, molecular and protein…
In this paper we study several classes of stochastic optimization algorithms enriched with heavy ball momentum. Among the methods studied are: stochastic gradient descent, stochastic Newton, stochastic proximal point and stochastic dual…
This article studies the solutions of time-dependent differential inclusions which is motivated by their utility in the modeling of certain physical systems. The differential inclusion is described by a time-dependent set-valued mapping…
This article investigates the problem of controlling linear time-invariant systems subject to time-varying and a priori unknown cost functions, state and input constraints, and exogenous disturbances. We combine the online convex…
Time-varying non-convex continuous-valued non-linear constrained optimization is a fundamental problem. We study conditions wherein a momentum-like regularising term allow for the tracking of local optima by considering an ordinary…
This work interprets and generalizes consensus-type algorithms as switching dynamics leading to symmetrization of some vector variables with respect to the actions of a finite group. We show how the symmetrization framework we develop…
Controlling complex dynamical systems has been a topic of considerable interest in academic circles in recent decades. While existing works have primarily focused on closed-loop control schemes with infinite-time durations, this paper…
In a Hilbert setting we aim to study a second order in time differential equation, combining viscous and Hessian-driven damping, containing a time scaling parameter function and a Tikhonov regularization term. The dynamical system is…
Minimax optimization has been central in addressing various applications in machine learning, game theory, and control theory. Prior literature has thus far mainly focused on studying such problems in the continuous domain, e.g.,…
Consider convex optimization problems subject to a large number of constraints. We focus on stochastic problems in which the objective takes the form of expected values and the feasible set is the intersection of a large number of convex…
Composition optimization is widely-applied in nonconvex machine learning. Various advanced stochastic algorithms that adopt momentum and variance reduction techniques have been developed for composition optimization. However, these…
In this paper a novel stochastic optimization and extremum seeking algorithm is presented, one which is based on time-delayed random perturbations and step size adaptation. For the case of a one-dimensional quadratic unconstrained…
Our work is part of the close link between continuous-time dissipative dynamical systems and optimization algorithms, and more precisely here, in the stochastic setting. We aim to study stochastic convex minimization problems through the…
We empirically evaluate the finite-time performance of several simulation-optimization algorithms on a testbed of problems with the goal of motivating further development of algorithms with strong finite-time performance. We investigate if…
In this paper, we consider a class of continuous-time, continuous-space stochastic optimal control problems. Building upon recent advances in Markov chain approximation methods and sampling-based algorithms for deterministic path planning,…
In this paper we analyze several new methods for solving nonconvex optimization problems with the objective function formed as a sum of two terms: one is nonconvex and smooth, and another is convex but simple and its structure is known.…