Related papers: Higher Order Force Gradient Symplectic Algorithms
The systematization of the purely Lagrangean approach to constrained systems in the form of an algorithm involves the iterative construction of a generalized Hessian matrix W taking a rectangular form. This Hessian will exhibit as many left…
Self-gravitational force calculation for infinitesimally thin disks is important for studies on the evolution of galactic and protoplanetary disks. Although high-order methods have been developed for hydrodynamic and magneto-hydrodynamic…
Bilevel optimization has been widely applied in many important machine learning applications such as hyperparameter optimization and meta-learning. Recently, several momentum-based algorithms have been proposed to solve bilevel optimization…
Rapid advances in data collection and processing capabilities have allowed for the use of increasingly complex models that give rise to nonconvex optimization problems. These formulations, however, can be arbitrarily difficult to solve in…
An appealing property of the natural gradient is that it is invariant to arbitrary differentiable reparameterizations of the model. However, this invariance property requires infinitesimal steps and is lost in practical implementations with…
We implement several symplectic integrators, which are based on two part splitting, for studying the chaotic behavior of one- and two-dimensional disordered Klein-Gordon lattices with many degrees of freedom and investigate their numerical…
Time-reversible symplectic methods, which are precisely compatible with Liouville's phase-volume-conservation theorem, are often recommended for computational simulations of Hamiltonian mechanics. Lack of energy drift is an apparent…
While SGD, which samples from the data with replacement is widely studied in theory, a variant called Random Reshuffling (RR) is more common in practice. RR iterates through random permutations of the dataset and has been shown to converge…
We established a new eighth-order iterative method, consisting of three steps, for solving nonlinear equations. Per iteration the method requires four evaluations (three function evaluations and one evaluation of the first derivative).…
A bi-level optimization framework (BiOPT) was proposed in [3] for convex composite optimization, which is a generalization of bi-level unconstrained minimization framework (BLUM) given in [20]. In this continuation paper, we introduce a…
Symplectic integrators evolve dynamical systems according to modified Hamiltonians whose error terms are also well-defined Hamiltonians. The error of the algorithm is the sum of each error Hamiltonian's perturbation on the exact solution.…
We design an algorithm which finds an $\epsilon$-approximate stationary point (with $\|\nabla F(x)\|\le \epsilon$) using $O(\epsilon^{-3})$ stochastic gradient and Hessian-vector products, matching guarantees that were previously available…
We provide new gradient-based methods for efficiently solving a broad class of ill-conditioned optimization problems. We consider the problem of minimizing a function $f : \mathbb{R}^d \rightarrow \mathbb{R}$ which is implicitly…
Solving quaternion kinematical differential equations is one of the most significant problems in the automation, navigation, aerospace and aeronautics literatures. Most existing approaches for this problem neither preserve the norm of…
Many differential equations with physical backgrounds are described as gradient systems, which are evolution equations driven by the gradient of some functionals, and such problems have energy conservation or dissipation properties. For…
Higher-order tensor methods were recently proposed for minimizing smooth convex and nonconvex functions. Higher-order algorithms accelerate the convergence of the classical first-order methods thanks to the higher-order derivatives used in…
This paper studies the complexity of finding an $\epsilon$-stationary point for stochastic bilevel optimization when the upper-level problem is nonconvex and the lower-level problem is strongly convex. Recent work proposed the first-order…
A novel symplectic algorithm is proposed to solve the Maxwell-Schr\"odinger (M-S) system for investigating light-matter interaction. Using the fourth-order symplectic integration and fourth-order collocated differences,…
We propose a first order algorithm, a modified version of FISTA, to solve an optimization problem with an objective function that is a sum of a possibly nonconvex function, with Lipschitz continuous gradient, and a convex function which can…
We derive an equivalent form of Halpern's fixed-point iteration scheme for solving a co-coercive equation (also called a root-finding problem), which can be viewed as a Nesterov's accelerated interpretation. We show that one method is…