Related papers: Revisiting a Fast Newton Solver for a 2-D Spectral…

Second-order optimization methods offer superior convergence rates but are often bottlenecked by the wall-clock cost of Hessian computation and factorization. In the moderate-dimensional regime where the full Hessian fits in memory,…

We consider the problem of reconstructing two signals from the autocorrelation and cross-correlation measurements. This inverse problem is a fundamental one in signal processing, and arises in many applications, including phase retrieval…

Indicator functions of taking values of zero or one are essential to numerous applications in machine learning and statistics. The corresponding primal optimization model has been researched in several recent works. However, its dual…

We develop a practical approach to semidefinite programming (SDP) that includes the von Neumann entropy, or an appropriate variant, as a regularization term. In particular we solve the dual of the regularized program, demonstrating how a…

We consider the closely related problems of bandit convex optimization with two-point feedback, and zero-order stochastic convex optimization with two function evaluations per round. We provide a simple algorithm and analysis which is…

Despite their popularity in the field of continuous optimisation, second-order quasi-Newton methods are challenging to apply in machine learning, as the Hessian matrix is intractably large. This computational burden is exacerbated by the…

Gradient descent is one of the most basic algorithms for solving continuous optimization problems. In [Jordan, PRL, 95(5):050501, 2005], Jordan proposed the first quantum algorithm for estimating gradients of functions close to linear, with…

This is the second in a series of papers on the numerical treatment of hyperelliptic theta-functions with spectral methods. A code for the numerical evaluation of solutions to the Ernst equation on hyperelliptic surfaces of genus 2 is…

The adaptive cubic regularization algorithm employing the inexact gradient and Hessian is proposed on general Riemannian manifolds, together with the iteration complexity to get an approximate second-order optimality under certain…

We present a new adaptive method for electronic structure calculations based on novel fast algorithms for reduction of multivariate mixtures. In our calculations, spatial orbitals are maintained as Gaussian mixtures whose terms are selected…

We derive a priori interior Hessian estimates and interior regularity for the $\sigma_2$ equation in dimension four. Our method provides respectively a new proof for the corresponding three dimensional results and a Hessian estimate for…

In this paper we accomplish the development of the fast rank-adaptive solver for tensor-structured symmetric positive definite linear systems in higher dimensions. In [arXiv:1301.6068] this problem is approached by alternating minimization…

For the Stokes equation over 2D and 3D domains, explicit a posteriori and a priori error estimation are novelly developed for the finite element solution. The difficulty in handling the divergence-free condition of the Stokes equation is…

Second-order methods for neural network optimization have several advantages over methods based on first-order gradient descent, including better scaling to large mini-batch sizes and fewer updates needed for convergence. But they are…

We consider minimizing a function consisting of a quadratic term and a proximable term which is possibly nonconvex and nonsmooth. This problem is also known as scaled proximal operator. Despite its simple form, existing methods suffer from…

This paper presents an efficient optimization technique for super-resolution two-dimensional (2D) direction of arrival (DOA) estimation by introducing a new formulation of atomic norm minimization (ANM). ANM allows gridless angle estimation…

The problem of finding a solution to the linear system $Ax = b$ with certain minimization properties arises in numerous scientific and engineering areas. In the era of big data, the stochastic optimization algorithms become increasingly…

We study the ubiquitous super-resolution problem, in which one aims at localizing positive point sources in an image, blurred by the point spread function of the imaging device. To recover the point sources, we propose to solve a convex…

Newton's method may exhibit slower convergence than vanilla Gradient Descent in its initial phase on strongly convex problems. Classical Newton-type multilevel methods mitigate this but, like Gradient Descent, achieve only linear…

Newton's method may exhibit slower convergence than vanilla Gradient Descent in its initial phase on strongly convex problems. Classical Newton-type multilevel methods mitigate this but, like Gradient Descent, achieve only linear…