Related papers: Continuous Newton-like Methods featuring Inertia a…
In this paper, we propose an inexact Newton-like conditional gradient method for solving constrained systems of nonlinear equations. The local convergence of the new method as well as results on its rate are established by using a general…
Inertial-based navigation refers to the navigation methods or systems that have inertial information or sensors as the core part and integrate a spectrum of other kinds of sensors for enhanced performance. Through a series of papers, the…
For a linear equality constrained convex optimization problem involving two objective functions with a ``nonsmooth" + ``nonsmooth" composite structure, we study two algorithms derived from a mixed-order dynamical system which incorporates…
Newton method is one of the most powerful methods for finding solutions of nonlinear equations and for proving their existence. In its "pure" form it has fast convergence near the solution, but small convergence domain. On the other hand…
We investigate stochastic resetting in coupled systems involving two degrees of freedom, where only one variable is reset. The resetting variable, which we think of as hidden, indirectly affects the remaining observable variable through…
We use differential equations based approaches to provide some {\it \textbf{physics}} insights into analyzing the dynamics of popular optimization algorithms in machine learning. In particular, we study gradient descent, proximal gradient…
Modified Newtonian dynamics can be considered as an effect derived from a squeezable extra dimension space. The third law of Newtonian dynamics can be managed to remain valid in the 5-space. The critical acceleration parameter $a_0$ appears…
We introduce new multilevel methods for solving large-scale unconstrained optimization problems. Specifically, the philosophy of multilevel methods is applied to Newton-type methods that regularize the Newton sub-problem using second order…
Two algorithms are proposed, analyzed, and tested for solving continuous optimization problems with nonlinear equality constraints. Each is an extension of a stochastic momentum-based method from the unconstrained setting to the setting of…
Second-order continuous-time dissipative dynamical systems with viscous and Hessian driven damping have inspired effective first-order algorithms for solving convex optimization problems. While preserving the fast convergence properties of…
A version of the Dynamical Systems Gradient Method for solving ill-posed nonlinear monotone operator equations is studied in this paper. A discrepancy principle is proposed and justified. A numerical experiment was carried out with the new…
Newton's method is the most widespread high-order method, demanding the gradient and the Hessian of the objective function. However, one of the main disadvantages of Newtons method is its lack of global convergence and high iteration cost.…
A new approach in the Newtonian space and time, based upon the assumption that inertial mass is the quantitative measure of the matter. It has been shown that in case of a special physical system, a supposed matter transfer may reproduce…
It has been shown in the past century that the particle inertia against velocity change has increased at higher velocity values. This increase has been predicted in principle in the framework of special theory of relativity. However, any…
Quasi-Newton methods form an important class of methods for solving nonlinear optimization problems. In such methods, first order information is used to approximate the second derivative. The aim is to mimic the fast convergence that can be…
This paper develops a comprehensive extension of the $\Lambda$-set framework for optimal control, introducing second-order $\Lambda$-sets and generalizing the theory to non-smooth, hybrid, and stochastic hybrid systems. We first establish…
Gradient descent-based optimization methods underpin the parameter training of neural networks, and hence comprise a significant component in the impressive test results found in a number of applications. Introducing stochasticity is key to…
We present the online Newton's method, a single-step second-order method for online nonconvex optimization. We analyze its performance and obtain a dynamic regret bound that is linear in the cumulative variation between round optima. We…
Optimization plays a key role in machine learning. Recently, stochastic second-order methods have attracted much attention due to their low computational cost in each iteration. However, these algorithms might perform poorly especially if…
This is the final paper in a series that introduces geodesic molecular dynamics at constant potential energy. This dynamics is entitled NVU dynamics in analogy to standard energy-conserving Newtonian NVE dynamics. In the first two papers…