Related papers: Modified projected Gauss-Newton method for constra…
We propose a stochastic variance-reduced cubic regularized Newton algorithm to optimize the finite-sum problem over a Riemannian submanifold of the Euclidean space. The proposed algorithm requires a full gradient and Hessian update at the…
Linear inverse problems are ubiquitous. Often the measurements do not follow a Gaussian distribution. Additionally, a model matrix with a large condition number can complicate the problem further by making it ill-posed. In this case, the…
This paper presents a stochastic block-coordinate proximal Newton method for minimizing the sum of a blockwise Lipschitz-continuously differentiable function and a separable nonsmooth convex function. At each iteration, the method randomly…
The variational approach to fracture is effective for simulating the nucleation and propagation of complex crack patterns, but is computationally demanding. The model is a strongly nonlinear non-convex variational inequality that demands…
We propose an energy-optimized invariant energy quadratization method to solve the gradient flow models in this paper, which requires only one linear energy-optimized step to correct the auxiliary variables on each time step. In addition to…
We investigate the nonlinear regression problem under L2 loss (square loss) functions. Traditional nonlinear regression models often result in non-convex optimization problems with respect to the parameter set. We show that a convex…
A new result in convex analysis on the calculation of proximity operators in certain scaled norms is derived. We describe efficient implementations of the proximity calculation for a useful class of functions; the implementations exploit…
We develop a novel switching dynamics that converges to the Karush-Kuhn-Tucker (KKT) point of a nonlinear optimisation problem. This new approach is particularly notable for its lower dimensionality compared to conventional primal-dual…
Recent advances in quantized compressed sensing and high-dimensional estimation have shown that signal recovery is even feasible under strong non-linear distortions in the observation process. An important characteristic of associated…
Leveraging nonlinear parametrizations for model reduction can overcome the Kolmogorov barrier that affects transport-dominated problems. In this work, we build on the reduced dynamics given by Neural Galerkin schemes and propose to…
The popular BFGS quasi-Newton minimization algorithm under reasonable conditions converges globally on smooth convex functions. This result was proved by Powell in 1976: we consider its implications for functions that are not smooth. In…
We introduce a quadratically-constrained approximation (QCAC) of the AC optimal power flow (AC-OPF) problem. Unlike existing approximations like the DC-OPF, our model does not rely on typical assumptions such as high reactance-to-resistance…
We study stochastic second-order methods for solving general non-convex optimization problems. We propose using a special version of momentum to stabilize the stochastic gradient and Hessian estimates in Newton's method. We show that…
In this paper we consider the iteratively regularized Gauss-Newton method for solving nonlinear ill-posed inverse problems. Under merely Lipschitz condition, we prove that this method together with an a posteriori stopping rule defines an…
The recently introduced structured input-output analysis is a powerful method for capturing nonlinear phenomena associated with incompressible flows, and this paper extends that method to the compressible regime. The proposed method relies…
This paper proposes a novel stochastic version of damped and regularized BFGS method for addressing the above problems.
This work proposes an implementable proximal-type method for a broad class of optimization problems involving nonsmooth and nonconvex objective and constraint functions. In contrast to existing methods that rely on an ad hoc model…
This work considers the nonconvex, nonsmooth problem of minimizing a composite objective of the form $f(g(x))+h(x)$ where the inner mapping $g$ is a smooth finite summation or expectation amenable to variance reduction. In such settings,…
We study finite-sum non-convex optimization $\min_{x\in\mathbb{R}^d} F(x) \;=\; \frac{1}{n}\sum_{i=1}^n f_i(x)$ and analyze a variance-reduced cubic Newton method based on EMA-smoothed SARAH estimators for both gradient and Hessian…
Functional constrained optimization is becoming more and more important in machine learning and operations research. Such problems have potential applications in risk-averse machine learning, semisupervised learning, and robust optimization…