Related papers: Adaptive Inertial Method
It is well-known that the lower bound of iteration complexity for solving nonconvex unconstrained optimization problems is $\Omega(1/\epsilon^2)$, which can be achieved by standard gradient descent algorithm when the objective function is…
We propose a new relative-error inexact version of the alternating direction method of multipliers (ADMM) for convex optimization. We prove the asymptotic convergence of our main algorithm as well as pointwise and ergodic…
We propose generic acceleration schemes for a wide class of optimization and iterative schemes based on relaxation and inertia. In particular, we introduce methods that automatically tunes the acceleration coefficients online, and establish…
In this paper, we propose a general extra-gradient scheme for solving monotone variational inequalities (VI), referred to here as Approximation-based Regularized Extra-gradient method (ARE). The first step of ARE solves a VI subproblem with…
The alternating minimization (AM) method is a fundamental method for minimizing convex functions whose variable consists of two blocks. How to efficiently solve each subproblems when applying the AM method is the most concerned task. In…
Consider composite nonconvex optimization problems where the objective function consists of a smooth nonconvex term (with Lipschitz-continuous gradient) and a convex (possibly nonsmooth) term. Existing parameter-free methods for such…
Network estimation from multi-variate point process or time series data is a problem of fundamental importance. Prior work has focused on parametric approaches that require a known parametric model, which makes estimation procedures less…
Proximal bundle methods (PBM) are a powerful class of algorithms for convex optimization. Compared to gradient descent, PBM constructs more accurate surrogate models that incorporate gradients and function values from multiple past…
We propose and analyze an Accelerated Rearrangement Method (ARM) for solving a class of nonconvex optimization problems involving two-phase composites. These problems include maximizing the (work) energy of a membrane governed by the…
In this paper, we propose objective-function-free (OFF) variants of the proximal Newton method for nonconvex composite optimization problems and the regularized Newton method for unconstrained optimization problems, respectively, using…
We propose a novel adaptive, accelerated algorithm for the stochastic constrained convex optimization setting. Our method, which is inspired by the Mirror-Prox method, \emph{simultaneously} achieves the optimal rates for smooth/non-smooth…
Efforts to achieve better accuracy in numerical relativity have so far focused either on implementing second order accurate adaptive mesh refinement or on defining higher order accurate differences and update schemes. Here, we argue for the…
We propose a new splitting method for strong numerical solution of the Cox-Ingersoll-Ross model. For this method, applied over both deterministic and adaptive random meshes, we prove a uniform moment bound and strong error results of order…
We provide improved convergence rates for various \emph{non-smooth} optimization problems via higher-order accelerated methods. In the case of $\ell_\infty$ regression, we achieves an $O(\epsilon^{-4/5})$ iteration complexity, breaking the…
As first-order optimization methods become the method of choice for solving large-scale optimization problems, optimization solvers based on first-order algorithms are being built. Such general-purpose solvers must robustly detect…
Adam-type optimizers, as a class of adaptive moment estimation methods with the exponential moving average scheme, have been successfully used in many applications of deep learning. Such methods are appealing due to the capability on…
We develop an interior-point approach to solve constrained variational inequality (cVI) problems. Inspired by the efficacy of the alternating direction method of multipliers (ADMM) method in the single-objective context, we generalize ADMM…
In this paper, we propose a unified framework of inexact stochastic Alternating Direction Method of Multipliers (ADMM) for solving nonconvex problems subject to linear constraints, whose objective comprises an average of finite-sum smooth…
This paper considers unconstrained convex optimization problems with time-varying objective functions. We propose algorithms with a discrete time-sampling scheme to find and track the solution trajectory based on prediction and correction…
An algorithm is devised for solving minimization problems with equality constraints. The algorithm uses first-order derivatives of both the objective function and the constraints. The step is computed as a sum between a steepest-descent…