Related papers: A Bregman inertial forward-reflected-backward meth…
This paper proposes two proximal Newton-CG methods for convex nonsmooth optimization problems in composite form. The algorithms are based on a a reformulation of the original nonsmooth problem as the unconstrained minimization of a…
This paper investigates the asymptotic behavior of a forward-backward-forward (FBF) type differential equation and its discrete counterpart for solving quasimonotone variational inequalities (VIs). Building on recent continuous-time…
Many important machine learning applications involve regularized nonconvex bi-level optimization. However, the existing gradient-based bi-level optimization algorithms cannot handle nonconvex or nonsmooth regularizers, and they suffer from…
We prove that the block-coordinate Frank-Wolfe (BCFW) algorithm converges with state-of-the-art rates in both convex and nonconvex settings under a very mild "block-iterative" assumption. This appears to be the first result on BCFW…
In this paper we present a variant of the proximal forward-backward splitting iteration for solving nonsmooth optimization problems in Hilbert spaces, when the objective function is the sum of two nondifferentiable convex functions. The…
This paper introduces a first-order majorization-minimization framework based on a high-order majorant for continuous functions, incorporating a non-quadratic regularization term of degree $p>1$. Notably, it is shown to be valid if and only…
We propose an optimization proxy in terms of iterative implicit gradient methods for solving constrained optimization problems with nonconvex loss functions. This framework can be applied to a broad range of machine learning settings,…
Minimization of a smooth function on a sphere or, more generally, on a smooth manifold, is the simplest non-convex optimization problem. It has a lot of applications. Our goal is to propose a version of the gradient projection algorithm for…
We propose two numerical algorithms in the fully nonconvex setting for the minimization of the sum of a smooth function and the composition of a nonsmooth function with a linear operator. The iterative schemes are formulated in the spirit…
We consider the problem of minimizing the composition of a smooth (nonconvex) function and a smooth vector mapping, where the inner mapping is in the form of an expectation over some random variable or a finite sum. We propose a stochastic…
Nonconvex optimization problems are widespread in modern machine learning and data science. We introduce an extrapolation strategy into a class of preconditioned second-order convex splitting algorithms for nonconvex optimization problems.…
This paper presents a unified analysis for the proximal subgradient method (Prox-SubGrad) type approach to minimize an overall objective of $f(x)+r(x)$, subject to convex constraints, where both $f$ and $r$ are weakly convex, nonsmooth, and…
We develop a novel stochastic primal dual splitting method with Bregman distances for solving a structured composite problems involving infimal convolutions in non-Euclidean spaces. The sublinear convergence in expectation of the…
In this work we study the method of Bregman projections for deterministic and stochastic convex feasibility problems with three types of control sequences for the selection of sets during the algorithmic procedure: greedy, random, and…
This work proposes an Accelerated Primal-Dual Fixed-Point (APDFP) method that employs Nesterov type acceleration to solve composite problems of the form min f(x) + g(Bx), where g is nonsmooth and B is a linear operator. The APDFP features…
We study optimization over Riemannian embedded submanifolds, where the objective function is relatively smooth in the ambient Euclidean space. Such problems have broad applications but are still largely unexplored. We introduce two…
The proximal inertial gradient descent is efficient for the composite minimization and applicable for broad of machine learning problems. In this paper, we revisit the computational complexity of this algorithm and present other novel…
We consider the method of quasi-solutions (also referred to as Ivanov regularization) for the regularization of linear ill-posed problems in non-reflexive Banach spaces. Using the equivalence to a metric projection onto the image of the…
We propose novel algorithms combining accelerated gradient flows with linearized projection-free treatments of non-convex constraints and BDF pseudo-temporal discretization for quadratic energy minimization. A general framework is developed…
The (global) Lipschitz smoothness condition is crucial in establishing the convergence theory for most optimization methods. Unfortunately, most machine learning and signal processing problems are not Lipschitz smooth. This motivates us to…