Related papers: An Element-wise RSAV Algorithm for Unconstrained O…
We exploit analogies between first-order algorithms for constrained optimization and non-smooth dynamical systems to design a new class of accelerated first-order algorithms for constrained optimization. Unlike Frank-Wolfe or projected…
In this paper, we propose a novel family of high-order numerical schemes for the gradient flow models based on the scalar auxiliary variable (SAV) approach, which is named the high-order scalar auxiliary variable (HSAV) method. The newly…
This article is concerned with the numerical solution of convex variational problems. More precisely, we develop an iterative minimisation technique which allows for the successive enrichment of an underlying discrete approximation space in…
In this paper, we present a second order, linear, fully decoupled, and unconditionally energy stable scheme for solving the Erickson-Leslie model. This approach integrates the pressure correction method with a scalar auxiliary variable…
It is well-known that the Allen-Cahn equation not only satisfies the energy dissipation law but also possesses the maximum bound principle (MBP) in the sense that the absolute value of its solution is pointwise bounded for all time by some…
A Riemannian gradient descent algorithm and a truncated variant are presented to solve systems of phaseless equations $|Ax|^2=y$. The algorithms are developed by exploiting the inherent low rank structure of the problem based on the…
We propose a general formulation of nonconvex and nonsmooth sparse optimization problems with convex set constraint, which can take into account most existing types of nonconvex sparsity-inducing terms, bringing strong applicability to a…
This work explores an extension of machine learning-optimized piecewise polynomial approximation by incorporating energy optimization as an additional objective. Traditional closed-form solutions enable continuity and approximation targets…
Non-convex optimization plays a key role in a growing number of machine learning applications. This motivates the identification of specialized structure that enables sharper theoretical analysis. One such identified structure is…
We consider a continuous-time optimization method based on a dynamical system, where a massive particle starting at rest moves in the conservative force field generated by the objective function, without any kind of friction. We formulate a…
This paper explores adaptive variance reduction methods for stochastic optimization based on the STORM technique. Existing adaptive extensions of STORM rely on strong assumptions like bounded gradients and bounded function values, or suffer…
Given a sample covariance matrix, we examine the problem of maximizing the variance explained by a linear combination of the input variables while constraining the number of nonzero coefficients in this combination. This is known as sparse…
The scalar auxiliary variable (SAV) approach of Shen et al. (2018), which presents a novel way to discretize a large class of gradient flows, has been extended and improved by many authors for general dissipative systems. In this work we…
We propose stochastic variance reduced algorithms for solving convex-concave saddle point problems, monotone variational inequalities, and monotone inclusions. Our framework applies to extragradient, forward-backward-forward, and…
We prove a general result demonstrating the power of Lagrangian relaxation in solving constrained maximization problems with arbitrary objective functions. This yields a unified approach for solving a wide class of {\em subset selection}…
We consider the problem of minimizing a sum of several convex non-smooth functions. We introduce a new algorithm called the selective linearization method, which iteratively linearizes all but one of the functions and employs simple…
This paper concerns an optimization algorithm for unconstrained non-convex problems where the objective function has sparse connections between the unknowns. The algorithm is based on applying a dissipation preserving numerical integrator,…
In this paper, we develop a symmetric accelerated stochastic Alternating Direction Method of Multipliers (SAS-ADMM) for solving separable convex optimization problems with linear constraints. The objective function is the sum of a possibly…
In this paper, we construct novel first- and second-order decoupled schemes for the Navier-Stokes equations based on the penalty method and the sequential regularization method (SRM), respectively. These schemes do not require the boundary…
Extremum seeking (ES) optimization approach has been very popular due to its non-model based analysis and implementation. This approach has been mostly used with gradient based search algorithms. Since least squares (LS) algorithms are…