Related papers: Second Order Threshold Dynamics Schemes for Two Ph…
Algorithms for bilevel optimization often encounter Hessian computations, which are prohibitive in high dimensions. While recent works offer first-order methods for unconstrained bilevel problems, the constrained setting remains relatively…
The single-step explicit time integration methods have long been valuable for solving large-scale nonlinear structural dynamic problems, classified into single-solve and multi-sub-step approaches. However, no existing explicit single-solve…
In this paper, we propose a unified two-phase scheme to accelerate any high-order regularized tensor approximation approach on the smooth part of a composite convex optimization model. The proposed scheme has the advantage of not needing to…
A modification of the standard Boris algorithm, called filtered Boris algorithm, is proposed for the numerical integration of the equations of motion of charged particles in a strong non-uniform magnetic field in the asymptotic scaling…
We establish rigorous \emph{a posteriori} error bounds for a space-time finite element method of arbitrary order discretising linear wave problems in second order formulation. The method combines standard finite elements in space and…
In this paper, we propose several new stochastic second-order algorithms for policy optimization that only require gradient and Hessian-vector product in each iteration, making them computationally efficient and comparable to policy…
The question of optimal spanwise-periodic modification for the stabilisation of spanwise-invariant flows is addressed. A 2nd-order sensitivity analysis is conducted for the linear temporal stability of parallel flows U0 subject to…
Subspace representation is a fundamental technique in various fields of machine learning. Analyzing a geometrical relationship among multiple subspaces is essential for understanding subspace series' temporal and/or spatial dynamics. This…
In this paper, we propose two novel non-stationary first-order primal-dual algorithms to solve nonsmooth composite convex optimization problems. Unlike existing primal-dual schemes where the parameters are often fixed, our methods use…
New implicit and implicit-explicit time-stepping methods for the wave equation in second-order form are described with application to two and three-dimensional problems discretized on overset grids. The implicit schemes are single step,…
Bilevel optimization has been widely applied in many important machine learning applications such as hyperparameter optimization and meta-learning. Recently, several momentum-based algorithms have been proposed to solve bilevel optimization…
We propose first order algorithms for convex optimization problems where the feasible set is described by a large number of convex inequalities that is to be explored by subgradient projections. The first algorithm is an adaptation of a…
Mean field approximation is a powerful technique which has been used in many settings to study large-scale stochastic systems. In the case of two-timescale systems, the approximation is obtained by a combination of scaling arguments and the…
Standard first-order stochastic optimization algorithms base their updates solely on the average mini-batch gradient, and it has been shown that tracking additional quantities such as the curvature can help de-sensitize common…
We establish lower bounds on the complexity of finding $\epsilon$-stationary points of smooth, non-convex high-dimensional functions using first-order methods. We prove that deterministic first-order methods, even applied to arbitrarily…
Neural stochastic differential equation model with a Brownian motion term can capture epistemic uncertainty of deep neural network from the perspective of a dynamical system. The goal of this paper is to improve the convergence rate of the…
Trust region and cubic regularization methods have demonstrated good performance in small scale non-convex optimization, showing the ability to escape from saddle points. Each iteration of these methods involves computation of gradient,…
We introduce a class of first-order methods for smooth constrained optimization that are based on an analogy to non-smooth dynamical systems. Two distinctive features of our approach are that (i) projections or optimizations over the entire…
The approximate solution of the Cauchy problem for second-order evolution equations is performed, first of all, using three-level time approximations. Such approximations are easily constructed and relatively uncomplicated to investigate…
The late-time phase-ordering kinetics of the O(n) model for a non-conserved order parameter are considered for the case where the O(n) symmetry is broken by the initial conditions or by an external field. An approximate theoretical…