Related papers: Dissipativity Theory for Nesterov's Accelerated Me…
Nesterov SGD is widely used for training modern neural networks and other machine learning models. Yet, its advantages over SGD have not been theoretically clarified. Indeed, as we show in our paper, both theoretically and empirically,…
Dissipativity is an input-output (IO) characterization of nonlinear systems that enables compositional robust control through Vidyasagar's Network Dissipativity Theorem (VDNT). However, determining the dissipativity of a system is an…
The main goal of this paper is to prove that if the energy-momentum (or energy-Casimir) method predicts formal instability of a relative equilibrium in a Hamiltonian system with symmetry, then with the addition of dissipation, the relative…
We formulate two classes of first-order algorithms more general than previously studied for minimizing smooth and strongly convex or, respectively, smooth and convex functions. We establish sufficient conditions, via new discrete Lyapunov…
Gradient restarting has been shown to improve the numerical performance of accelerated gradient methods. This paper provides a mathematical analysis to understand these advantages. First, we establish global linear convergence guarantees…
This paper deals with speeding up the convergence of a class of two-step iterative methods for solving linear systems of equations. To implement the acceleration technique, the residual norm associated with computed approximations for each…
In this paper, we generalize the well-known Nesterov's accelerated gradient (AG) method, originally designed for convex smooth optimization, to solve nonconvex and possibly stochastic optimization problems. We demonstrate that by properly…
In this paper we study connections between structured storage or Lyapunov functions of a class of interconnected systems (dynamical networks) and dissipativity properties of the individual systems. We prove that if a dynamical network,…
We study the instability properties of Nesterov's ODE in non-conservative settings, where the driving term is not necessarily the gradient of a potential function. While convergence properties under Nesterov's ODE are well-characterized for…
We investigate the strong convergence properties of a Nesterov type algorithm with two Tikhonov regularization terms in connection to the minimization problem of a smooth convex function $f.$ We show that the generated sequences converge…
In this paper we deal with a general second order continuous dynamical system associated to a convex minimization problem with a Fr\`echet differentiable objective function. We show that inertial algorithms, such as Nesterov's algorithm,…
In this paper, we develop a unified framework able to certify both exponential and subexponential convergence rates for a wide range of iterative first-order optimization algorithms. To this end, we construct a family of parameter-dependent…
We adress the problem of interactions between the longitudinal velocity increment and the energy dissipation rate in fully developed turbulence. The coupling between these two quantities is experimentally investigated by the theory of…
This paper develops a variational inference framework for control of infinite dimensional stochastic systems. We employ a measure theoretic approach which relies on the generalization of Girsanov's theorem, as well as the relation between…
The main goal of the paper is to define and use a condition sufficient to choose a unique solution to conservation law systems with a singular measure in initial data. Different approximations can lead to solutions with different…
This paper considers the problem of decentralized analysis and control synthesis to verify and ensure properties like stability and dissipativity of a large-scale networked system comprised of linear subsystems interconnected in an…
This paper considers the distributed optimization problem over a network, where the objective is to optimize a global function formed by a sum of local functions, using only local computation and communication. We develop an Accelerated…
In this paper we propose a new sequential data assimilation method for non-linear ordinary differential equations with compact state space. The method is designed so that the Lyapunov exponents of the corresponding estimation error dynamics…
We derive statistical-mechanical speed limits on dissipation from the classical, chaotic dynamics of many-particle systems. In one, the rate of irreversible entropy production in the environment is the maximum speed of a deterministic…
In this paper we consider the problem of computing the stationary distribution of nearly completely decomposable Markov processes, a well-established area in the classical theory of Markov processes with broad applications in the design,…