Related papers: From Adam to Adam-Like Lagrangians: Second-Order N…
We introduce four original concepts: First, the point-like object (PO) specified as a classical extended real object whose response to an external force is aptly specified solely by the trajectory of a single point, whose velocity…
This paper is a continuation of [13], where new variational principles were introduced based on the concept of anti-selfdual (ASD) Lagrangians. We continue here the program of using these Lagrangians to provide variational formulations and…
Research into optimisation for deep learning is characterised by a tension between the computational efficiency of first-order, gradient-based methods (such as SGD and Adam) and the theoretical efficiency of second-order, curvature-based…
We consider a class of finite-dimensional dynamical systems whose equations of motion are derived from a non-local-in-time action principle. The action functional has a zeroth order piece derived from a local Hamiltonian and a perturbation…
This paper is devoted to the study of acceleration methods for an inequality constrained convex optimization problem by using Lyapunov functions. We first approximate such a problem as an unconstrained optimization problem by employing the…
This study investigates the potential for biological systems to be governed by a variational principle, suggesting that such systems may evolve to minimize or optimize specific quantities. To explore this idea, we focus on identifying…
We describe a recipe to generate "nonlocal" constants of motion for ODE Lagrangian systems. As a sample application, we recall a nonlocal constant of motion for dissipative mechanical systems, from which we can deduce global existence and…
Adaptive gradient methods, e.g. \textsc{Adam}, have achieved tremendous success in machine learning. Scaling the learning rate element-wisely by a certain form of second moment estimate of gradients, such methods are able to attain rapid…
The need to develop models to predict the motion of microrobots, or robots of a much smaller scale, moving in fluids in a low Reynolds number regime, and in particular, in non Newtonian fluids, cannot be understated. The article develops a…
Nonlinear dynamical systems are ubiquitous in nature and they are hard to forecast. Not only they may be sensitive to small perturbations in their initial conditions, but they are often composed of processes acting at multiple scales.…
A Lagrangian relativistic approach to the non--linear dynamics of cosmological perturbations of an irrotational collisionless fluid is considered. Solutions are given at second order in perturbation theory for the relevant fluid and…
In this paper, we present new results on finite- and fixed-time convergence for dynamical systems using LaSalle-like invariance principles. In particular, we provide first and second-order non-smooth Lyapunov-like results for finite- and…
We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. The method is straightforward to implement, is computationally efficient, has…
We identity a by-far-unrecognized problem of Adam-style optimizers which results from unnecessary coupling between momentum and adaptivity. The coupling leads to instability and divergence when the momentum and adaptivity parameters are…
In this paper, we study a class of convex composite optimization problems. We begin by characterizing the equivalence between the primal/dual strong second-order sufficient condition and the dual/primal nondegeneracy condition. Building on…
We develop an explicit, second-order, variational time integrator for full body dynamics that preserves the momenta of the continuous dynamics, such as linear and angular momenta, and exhibits near-conservation of total energy over…
This paper deals with the time evolution in the matter era of perturbations in Friedman-Lemaitre models with arbitrary density parameter $\Omega$, with either a zero cosmological constant, $\Lambda = 0$, or with a non-zero cosmological…
A general non-local point transformation for position-dependent mass Lagrangians and their mapping into a "constant unit-mass" Lagrangians in the generalized coordinates is introduced. The conditions on the invariance of the related…
Recently, there has been great interest in connections between continuous-time dynamical systems and optimization methods, notably in the context of accelerated methods for smooth and unconstrained problems. In this paper we extend this…
This work aims to minimize a continuously differentiable convex function with Lipschitz continuous gradient under linear equality constraints. The proposed inertial algorithm results from the discretization of the second-order primal-dual…