Related papers: Optimal First-Order Algorithms as a Function of In…
Algorithms for continuous optimization problems have a rich history of design and innovation over the past several decades, in which mathematical analysis of their convergence and complexity properties plays a central role. Besides their…
In this paper, we propose a new Fully Composite Formulation of convex optimization problems. It includes, as a particular case, the problems with functional constraints, max-type minimization problems, and problems of Composite…
This paper presents the formulation and analysis of a novel distributed maximum likelihood algorithm that utilizes a first-order optimization scheme. The proposed approach utilizes a static average consensus algorithm to reach agreement on…
We consider some classical optimization problems in path planning and network transport, and we introduce new auction-based algorithms for their optimal and suboptimal solution. The algorithms are based on mathematical ideas that are…
We consider unconstrained randomized optimization of convex objective functions. We analyze the Random Pursuit algorithm, which iteratively computes an approximate solution to the optimization problem by repeated optimization over a…
Majorization-minimization algorithms consist of successively minimizing a sequence of upper bounds of the objective function. These upper bounds are tight at the current estimate, and each iteration monotonically drives the objective…
We present an algorithm based on continuation techniques that can be applied to solve numerically minimization problems with equality constraints. We focus on problems with a great number of local minima which are hard to obtain by local…
In this paper we focus on the solution of online problems with time-varying, linear equality and inequality constraints. Our approach is to design a novel online algorithm by leveraging the tools of control theory. In particular, for the…
We present a practical and powerful new framework for both unconstrained and constrained submodular function optimization based on discrete semidifferentials (sub- and super-differentials). The resulting algorithms, which repeatedly compute…
A new algorithm for one-dimensional minimization is described in detail and the results of some tests on practical cases are reported and illustrated. The method requires only punctual computation of the function, and is suitable to be…
The research area of algorithms with predictions has seen recent success showing how to incorporate machine learning into algorithm design to improve performance when the predictions are correct, while retaining worst-case guarantees when…
Many recent studies on first-order methods (FOMs) focus on \emph{composite non-convex non-smooth} optimization with linear and/or nonlinear function constraints. Upper (or worst-case) complexity bounds have been established for these…
Optimization problems in engineering and applied mathematics are typically solved in an iterative fashion, by systematically adjusting the variables of interest until an adequate solution is found. The iterative algorithms that govern these…
Balancing convergence speed, generalization capability, and computational efficiency remains a core challenge in deep learning optimization. First-order gradient descent methods, epitomized by stochastic gradient descent (SGD) and Adam,…
This work studies constrained stochastic optimization problems where the objective and constraint functions are convex and expressed as compositions of stochastic functions. The problem arises in the context of fair classification, fair…
We introduce and analyze a new family of first-order optimization algorithms which generalizes and unifies both mirror descent and dual averaging. Within the framework of this family, we define new algorithms for constrained optimization…
Submodular function minimization is a fundamental optimization problem that arises in several applications in machine learning and computer vision. The problem is known to be solvable in polynomial time, but general purpose algorithms have…
This paper shows that the OSGA algorithm -- which uses first-order information to solve convex optimization problems with optimal complexity -- can be used to efficiently solve arbitrary bound-constrained convex optimization problems. This…
We provide a generalization of first-order necessary conditions of optimality for infinite-dimensional optimization problems with a finite number of inequality constraints and with a finite number of inequality and equality constraints. Our…
In the paper, we propose solving optimization problems (OPs) and understanding the Newton method from the optimal control view. We propose a new optimization algorithm based on the optimal control problem (OCP). The algorithm features…