Related papers: Fast convex optimization via inertial dynamics wit…
We propose notions of minimax and viscosity solutions for a class of fully nonlinear path-dependent PDEs with nonlinear, monotone, and coercive operators on Hilbert space. Our main result is well-posedness (existence, uniqueness, and…
Second-order optimization methods exhibit fast convergence to critical points, however, in nonconvex optimization, these methods often require restrictive step-sizes to ensure a monotonically decreasing objective function. In the presence…
Many data-fitting applications require the solution of an optimization problem involving a sum of large number of functions of high dimensional parameter. Here, we consider the problem of minimizing a sum of $n$ functions over a convex…
We develop and analyze several different second-order algorithms for computing a near-optimal solution path of a convex parametric optimization problem with smooth Hessian. Our algorithms are inspired by a differential equation perspective…
To solve convex optimization problems with a noisy gradient input, we analyze the global behavior of subgradient-like flows under stochastic errors. The objective function is composite, being equal to the sum of two convex functions, one…
In this paper, for a convex-concave bilinear saddle point problem, we propose a Tikhonov regularized second-order primal-dual dynamical system with slow damping, extrapolation and general time scaling parameters. Depending on the vanishing…
We consider the long time behavior of the solutions to the Burgers-FKPP equation with advection of a strength $\beta\in\mathbb{R}$. This equation exhibits a transition from pulled to pushed front behavior at $\beta_c=2$. We prove…
We propose some algorithms to find local minima in nonconvex optimization and to obtain global minima in some degree from the Newton Second Law without friction. With the key observation of the velocity observable and controllable in the…
In this paper, we propose an accelerated version for the Sinkhorn algorithm, which is the reference method for computing the solution to Entropic Optimal Transport. Its main draw-back is the exponential slow-down of convergence as the…
In this paper, we derive a Fast Reflected Forward-Backward (Fast RFB) algorithm to solve the problem of finding a zero of the sum of a maximally monotone operator and a monotone and Lipschitz continuous operator in a real Hilbert space. Our…
Natural policy gradient (NPG) and its variants are widely-used policy search methods in reinforcement learning. Inspired by prior work, a new NPG variant coined NPG-HM is developed in this paper, which utilizes the Hessian-aided momentum…
Fitting a function by using linear combinations of a large number $N$ of `simple' components is one of the most fruitful ideas in statistical learning. This idea lies at the core of a variety of methods, from two-layer neural networks to…
We investigate the strong and the weak convergence properties of the following gradient projection algorithm with Tikhonov regularizing term \[ x_{n+1}=P_{Q}(x_{n}-\gamma_{n}\nabla f(x_{n})-\gamma_{n}\alpha_{n}\nabla \phi (x_{n})), \] where…
We show that, for finite-sum minimization problems, incorporating partial second-order information of the objective function can dramatically improve the robustness to mini-batch size of variance-reduced stochastic gradient methods, making…
We consider the convergence of kinetic Langevin dynamics to its ergodic invariant measure, which is Gibbs distribution. Instead of the standard setup where the friction coefficient is a constant scalar, we investigate position-dependent…
We study the problem of sampling from a target distribution $\pi(q)\propto e^{-U(q)}$ on $\mathbb{R}^d$, where $U$ can be non-convex, via the Hessian-free high-resolution (HFHR) dynamics, which is a second-order Langevin-type process that…
We provide a framework for computing the exact worst-case performance of any algorithm belonging to a broad class of oracle-based first-order methods for composite convex optimization, including those performing explicit, projected,…
Obtaining lightweight and accurate approximations of Hessian applies in inverse problems governed by partial differential equations (PDEs) is an essential task to make both deterministic and Bayesian statistical large-scale inverse problems…
In this paper, an efficient modified Newton type algorithm is proposed for nonlinear unconstrianed optimization problems. The modified Hessian is a convex combination of the identity matrix (for steepest descent algorithm) and the Hessian…
We introduce a new dynamical system, at the interface between second-order dynamics with inertia and Newton's method. This system extends the class of inertial Newton-like dynamics by featuring a time-dependent parameter in front of the…