Related papers: Algorithmic approaches to avoiding bad local minim…
This article explores distributed convex optimization with globally-coupled constraints, where the objective function is a general nonsmooth convex function, the constraints include nonlinear inequalities and affine equalities, and the…
This paper proposes a novel technique called "successive stochastic smoothing" that optimizes nonsmooth and discontinuous functions while considering various constraints. Our methodology enables local and global optimization, making it a…
For a linear equality constrained convex optimization problem involving two objective functions with a ``nonsmooth" + ``nonsmooth" composite structure, we study two algorithms derived from a mixed-order dynamical system which incorporates…
In this paper we consider large-scale smooth optimization problems with multiple linear coupled constraints. Due to the non-separability of the constraints, arbitrary random sketching would not be guaranteed to work. Thus, we first…
Matrix factorization is a popular approach for large-scale matrix completion. The optimization formulation based on matrix factorization can be solved very efficiently by standard algorithms in practice. However, due to the non-convexity…
Nonlinear convex problems arise in various areas of applied mathematics and engineering. Classical techniques such as the relaxed proximal point algorithm (PPA) and the prediction correction (PC) method were proposed for linearly…
Minimax optimization has become a central tool in machine learning with applications in robust optimization, reinforcement learning, GANs, etc. These applications are often nonconvex-nonconcave, but the existing theory is unable to identify…
We study global optimization of non-convex functions through optimal control theory. Our main result establishes that (quasi-)optimal trajectories of a discounted control problem converge globally and practically asymptotically to the set…
Various optimal gradient-based algorithms have been developed for smooth nonconvex optimization. However, many nonconvex machine learning problems do not belong to the class of smooth functions and therefore the existing algorithms are…
A major obstacle to non-convex optimization is the problem of getting stuck in local minima. We introduce a novel metaheuristic to handle this issue, creating an alternate Hamiltonian that shares minima with the original Hamiltonian only…
Most non-convex optimization theory is built around gradient dynamics, leaving global convergence largely unexplored. The dominant paradigm focuses on stationarity, certifying only that the gradient norm vanishes, which is often a weak…
We introduce and study a geometric modification of the Douglas-Rach\-ford method called the Circumcentered-Douglas-Rachford method. This method iterates by taking the intersection of bisectors of reflection steps for solving certain classes…
We consider the problem of minimizing the sum of an average function of a large number of smooth convex components and a general, possibly non-differentiable, convex function. Although many methods have been proposed to solve this problem…
This paper presents a novel method for reformulating non-differentiable collision avoidance constraints into smooth nonlinear constraints using strong duality of convex optimization. We focus on a controlled object whose goal is to avoid…
Approximations of optimization problems arise in computational procedures and sensitivity analysis. The resulting effect on solutions can be significant, with even small approximations of components of a problem translating into large…
We analyse the behaviour of the newly introduced cyclic Douglas-Rachford algorithm for finding a point in the intersection of a finite number of closed convex sets. This work considers the case in which the target intersection set is…
This paper deals with convex nonsmooth optimization problems. We introduce a general smooth approximation framework for the original function and apply random (accelerated) coordinate descent methods for minimizing the corresponding smooth…
Non-convex optimization with local search heuristics has been widely used in machine learning, achieving many state-of-art results. It becomes increasingly important to understand why they can work for these NP-hard problems on typical…
The Douglas-Rachford algorithm is a classical and powerful splitting method for minimizing the sum of two convex functions and, more generally, finding a zero of the sum of two maximally monotone operators. Although this algorithm is well…
We provide novel theoretical results regarding local optima of regularized $M$-estimators, allowing for nonconvexity in both loss and penalty functions. Under restricted strong convexity on the loss and suitable regularity conditions on the…