Related papers: Best-first Search Algorithm for Non-convex Sparse …
This paper provides a comprehensive study of the nonmonotone forward-backward splitting (FBS) method for solving a class of nonsmooth composite problems in Hilbert spaces. The objective function is the sum of a Fr\'echet differentiable (not…
This paper proposes and analyzes a communication-efficient distributed optimization framework for general nonconvex nonsmooth signal processing and machine learning problems under an asynchronous protocol. At each iteration, worker machines…
We develop a family of accelerated stochastic algorithms that minimize sums of convex functions. Our algorithms improve upon the fastest running time for empirical risk minimization (ERM), and in particular linear least-squares regression,…
When equipped with efficient optimization algorithms, the over-parameterized neural networks have demonstrated high level of performance even though the loss function is non-convex and non-smooth. While many works have been focusing on…
In this paper we propose a second--order method for solving \emph{linear composite sparse optimization problems} consisting of minimizing the sum of a differentiable (possibly nonconvex function) and a nondifferentiable convex term. The…
It was recently established that for convex optimization problems with sparse optimal solutions (be it entry-wise sparsity or matrix rank-wise sparsity) it is possible to design first-order methods with linear convergence rates that depend…
An algorithm is proposed, analyzed, and tested experimentally for solving stochastic optimization problems in which the decision variables are constrained to satisfy equations defined by deterministic, smooth, and nonlinear functions. It is…
In this paper, we address the problem of minimizing a convex function f over a convex set, with the extra constraint that some variables must be integer. This problem, even when f is a piecewise linear function, is NP-hard. We study an…
We consider the problem of finding local minimizers in non-convex and non-smooth optimization. Under the assumption of strict saddle points, positive results have been derived for first-order methods. We present the first known results for…
In this work, we consider the distributed optimization of non-smooth convex functions using a network of computing units. We investigate this problem under two regularity assumptions: (1) the Lipschitz continuity of the global objective…
Nonnegative matrix factorization (NMF) has become a very popular technique in machine learning because it automatically extracts meaningful features through a sparse and part-based representation. However, NMF has the drawback of being…
Compressed sensing of simultaneously sparse and low-rank matrices enables recovery of sparse signals from a few linear measurements of their bilinear form. One important question is how many measurements are needed for a stable…
We study a class of stochastic nonconvex optimization in the form of $\min_{x\in\mathcal{X}} F(x):=\mathbb{E}_\xi [f(\phi(x,\xi))]$, i.e., $F$ is a composition of a convex function $f$ and a random function $\phi$. Leveraging an (implicit)…
We investigate quasi-Newton methods for minimizing a strictly convex quadratic function which is subject to errors in the evaluation of the gradients. The methods all give identical behavior in exact arithmetic, generating minimizers of…
As enjoying the closed form solution, least squares support vector machine (LSSVM) has been widely used for classification and regression problems having the comparable performance with other types of SVMs. However, LSSVM has two drawbacks:…
We study inexact fixed-point proximity algorithms for solving a class of sparse regularization problems involving the $\ell_0$ norm. Specifically, the $\ell_0$ model has an objective function that is the sum of a convex fidelity term and a…
A framework based on iterative coordinate minimization (CM) is developed for stochastic convex optimization. Given that exact coordinate minimization is impossible due to the unknown stochastic nature of the objective function, the crux of…
For the binary classification problem, a novel nonlinear kernel-free quadratic hyper-surface support vector machine with 0-1 loss function (QSSVM$_{0/1}$) is proposed. Specifically, the task of QSSVM$_{0/1}$ is to seek a quadratic…
We consider the minimization of submodular functions subject to ordering constraints. We show that this optimization problem can be cast as a convex optimization problem on a space of uni-dimensional measures, with ordering constraints…
We consider the fundamental problem in non-convex optimization of efficiently reaching a stationary point. In contrast to the convex case, in the long history of this basic problem, the only known theoretical results on first-order…