Related papers: Optimization of Inf-Convolution Regularized Noncon…
Regularized empirical risk minimization with constrained labels (in contrast to fixed labels) is a remarkably general abstraction of learning. For common loss and regularization functions, this optimization problem assumes the form of a…
In this paper, an inexact proximal-point penalty method is studied for constrained optimization problems, where the objective function is non-convex, and the constraint functions can also be non-convex. The proposed method approximately…
Non-convex optimization problems are ubiquitous in machine learning, especially in Deep Learning. While such complex problems can often be successfully optimized in practice by using stochastic gradient descent (SGD), theoretical analysis…
We consider a general optimization problem of minimizing a composite objective functional defined over a class of probability distributions. The objective is composed of two functionals: one is assumed to possess the variational…
We provide novel theoretical results regarding local optima of regularized $M$-estimators, allowing for nonconvexity in both loss and penalty functions. Under restricted strong convexity on the loss and suitable regularity conditions on the…
We tackle highly nonconvex, nonsmooth composite optimization problems whose objectives comprise a Moreau-Yosida regularized term. Classical nonconvex proximal splitting algorithms, such as nonconvex ADMM, suffer from lack of convergence for…
We propose a variable smoothing algorithm for solving nonconvexly constrained nonsmooth optimization problems. The target problem has two issues that need to be addressed: (i) the nonconvex constraint and (ii) the nonsmooth term. To handle…
Based on G. Lan's accelerated gradient sliding and general relation between the smoothness and strong convexity parameters of function under Legendre transformation we show that under rather general conditions the best known bounds for…
We consider the problem of minimizing the sum of a smooth function $h$ with a bounded Hessian, and a nonsmooth function. We assume that the latter function is a composition of a proper closed function $P$ and a surjective linear map $\cal…
The convex envelopes of the direct discrete measures, for the sparsity of vectors or for the low-rankness of matrices, have been utilized extensively as practical penalties in order to compute a globally optimal solution of the…
This paper presents a novel framework for nonconvex enhancement of minimization induced (MI) penalties while preserving the overall convexity of associated regularization models. MI penalties enable the adaptation to certain signal…
This work is concerned with the optimization of nonconvex, nonsmooth composite optimization problems, whose objective is a composition of a nonlinear mapping and a nonsmooth nonconvex function, that can be written as an infimal convolution…
We investigate a class of composite nonconvex functions, where the outer function is the sum of univariate extended-real-valued convex functions and the inner function is the limit of difference-of-convex functions. A notable feature of…
Minimax optimization has become a central tool in machine learning with applications in robust optimization, reinforcement learning, GANs, etc. These applications are often nonconvex-nonconcave, but the existing theory is unable to identify…
We consider a class of infinite-dimensional optimization problems in which a distributed vector-valued variable should pointwise almost everywhere take values from a given finite set $\mathcal{M}\subset\mathbb{R}^m$. Such hybrid…
In this paper, we study the local convergence of the standard ADMM scheme for a class of nonconvex composite problems arising from modern imaging and machine learning models. This problem is constrained by a closed convex set, while its…
Most algorithms for solving optimization problems or finding saddle points of convex-concave functions are fixed-point algorithms. In this work we consider the generic problem of finding a fixed point of an average of operators, or an…
We consider the problem of maximizing submodular functions; while this problem is known to be NP-hard, several numerically efficient local search techniques with approximation guarantees are available. In this paper, we propose a novel…
We present the viewpoint that optimization problems encountered in machine learning can often be interpreted as minimizing a convex functional over a function space, but with a non-convex constraint set introduced by model parameterization.…
We propose a nonconvexly regularized convex model for linear regression problems under non-Gaussian noise. The cost function of the proposed model is designed with a possibly non-quadratic data fidelity term and a nonconvex regularizer via…