Related papers: On the Instability of Nesterov's ODE under Non-Con…

The theoretical justification of the Hybrid Monte Carlo algorithm depends upon the molecular dynamics trajectories within it being exactly reversible. If computations were carried out with exact arithmetic then it would be easy to ensure…

A monotone self-mapping of the nonnegative orthant induces a monotone discrete-time dynamical system which evolves on the same orthant. If with respect to this system the origin is attractive then there must exists points whose image under…

We revisit the Ravine method of Gelfand and Tsetlin from a dynamical system perspective, study its convergence properties, and highlight its similarities and differences with the Nesterov accelerated gradient method. The two methods are…

We show that accelerated gradient descent, averaged gradient descent and the heavy-ball method for non-strongly-convex problems may be reformulated as constant parameter second-order difference equation algorithms, where stability of the…

The discovery of conservation laws is a cornerstone of scientific progress. However, identifying these invariants from observational data remains a significant challenge. We propose a hybrid framework to automate the discovery of conserved…

In this paper, we address the stability of transport systems and wave propagation on networks with time-varying parameters. We do so by reformulating these systems as non-autonomous difference equations and by providing a suitable…

We show that imposition of non-periodic, in place of periodic, boundary conditions (BC) can alter stability of modes in the Method of characteristics (MoC) employing certain ordinary-differential equation (ODE) numerical solvers. Thus,…

We examine the instabilities of a confined active nematic subjected to an orienting field using a low Reynolds number Ericksen-Leslie framework with active stresses and field-induced torques. Linear analysis reveals two distinct modes, with…

The choice of how to retain information about past gradients dramatically affects the convergence properties of state-of-the-art stochastic optimization methods, such as Heavy-ball, Nesterov's momentum, RMSprop and Adam. Building on this…

Stability analysis and control of linear impulsive systems is addressed in a hybrid framework, through the use of continuous-time time-varying discontinuous Lyapunov functions. Necessary and sufficient conditions for stability of impulsive…

Several recent works in online optimization and game dynamics have established strong negative complexity results including the formal emergence of instability and chaos even in small such settings, e.g., $2\times 2$ games. These results…

Lookahead-based acceleration methods, such as Nesterov's momentum, are widely used in optimization, but they often become unreliable in deep learning training mainly due to stochastic gradient noise and non-convex loss landscapes. In…

There appears to be a longtime, very slowly evolving state in dense simple fluids which, for high enough density, approaches a glassy nonergodic state. The nature of the nonergodic state can be characterized by the associated static…

We propose a data-driven framework for learning reduced-order moment dynamics from PDE-governed systems using Neural ODEs. In contrast to derivative-based methods like SINDy, which necessitate densely sampled data and are sensitive to…

We investigate the dynamical formation of nonlinear patterns in one-dimensional ring condensates under bichromatic periodic modulation of the interaction strength. The stability phase diagram of the condensate's homogeneous density state is…

Random reshuffling with momentum (RRM) corresponds to the SGD optimizer with momentum option enabled, as found in many machine learning libraries like PyTorch and TensorFlow. Despite its widespread use, the convergence properties of RRM do…

We present a unifying Nearly Asymptotically Invariant Manifold (NAIM) framework for understanding Nesterovs Accelerated Gradient (NAG) method. By lifting the first-order gradient flow into a second-order phase space we construct a NAIM a…

A central challenge in physics is to describe non-equilibrium systems driven by randomness, such as a randomly growing interface, or fluids subject to random fluctuations that account e.g. for local stresses and heat fluxes not related to…

We present a new accelerated stochastic second-order method that is robust to both gradient and Hessian inexactness, which occurs typically in machine learning. We establish theoretical lower bounds and prove that our algorithm achieves…

Recent years have seen a growing interest in understanding acceleration methods through the lens of ordinary differential equations (ODEs). Despite the theoretical advancements, translating the rapid convergence observed in continuous-time…