Related papers: Polling Set Construction and Worst-Case Complexity…

We develop a sketching algorithm to find the point on the convex hull of a dataset, closest to a query point outside it. Studying the convex hull of datasets can provide useful information about their geometric structure and their…

Convex optimization problems arising in applications often have favorable objective functions and complicated constraints, thereby precluding first-order methods from being immediately applicable. We describe an approach that exchanges the…

The nonlinear conjugate gradient methods are known to be an effective approach for standard unconstrained optimization problems especially for large-scale problems. This paper proposes a proximal nonlinear conjugate gradient method, which…

Random search methods are widely used for global optimization due to their theoretical generality and implementation simplicity. This paper proposes a depth-first directional search (DFDS) algorithm for globally solving nonconvex…

This document introduces a strategy to solve linear optimization problems. The strategy is based on the bounding condition each constraint produces on each one of the problem's dimension. The solution of a linear optimization problem is…

Optimization problems with set-valued objective functions arise in contexts such as multi-stage optimization with vector-valued objectives. The aim is to identify an optimizer -- a feasible point with an optimal objective value -- based on…

Projections onto sets are used in a wide variety of methods in optimization theory but not every method that uses projections really belongs to the class of projection methods as we mean it here. Here projection methods are iterative…

There has been much recent interest in finding unconstrained local minima of smooth functions, due in part of the prevalence of such problems in machine learning and robust statistics. A particular focus is algorithms with good complexity…

The constrained minimization (respectively maximization) of directed distances and of related generalized entropies is a fundamental task in information theory as well as in the adjacent fields of statistics, machine learning, artificial…

This letter presents a new spectral-clustering-based approach to the subspace clustering problem. Underpinning the proposed method is a convex program for optimal direction search, which for each data point d finds an optimal direction in…

Iteration methods based on barycentric rational interpolation are derived that exhibit accelerating orders of convergence. For univariate root search, the derivative-free methods approach quadratic convergence and the first-derivative…

Recent years have seen a flurry of activities in designing provably efficient nonconvex procedures for solving statistical estimation problems. Due to the highly nonconvex nature of the empirical loss, state-of-the-art procedures often…

Diffusion models have gained prominence as powerful generative tools for solving inverse problems due to their ability to model complex data distributions. However, existing methods typically rely on complete knowledge of the forward…

Two new optimization techniques based on projections onto convex space (POCS) framework for solving convex and some non-convex optimization problems are presented. The dimension of the minimization problem is lifted by one and sets…

We consider minimizing high-dimensional smooth nonconvex objectives using only noisy pairwise comparisons. Unlike classical zeroth-order methods limited by the ambient dimension $d$, we propose Noisy-Comparison Random Search (NCRS), a…

Convex sparsity-inducing regularizations are ubiquitous in high-dimensional machine learning, but solving the resulting optimization problems can be slow. To accelerate solvers, state-of-the-art approaches consist in reducing the size of…

We introduce a search problem generalizing the typical setting of Binary Search on the line. Similar to the setting for Binary Search, a target is chosen adversarially on the line, and in response to a query, the algorithm learns whether…

The geometric problem of estimating an unknown compact convex set from evaluations of its support function arises in a range of scientific and engineering applications. Traditional approaches typically rely on estimators that minimize the…

Recent progress in deep learning has been driven by increasingly larger models. However, their computational and energy demands have grown proportionally, creating significant barriers to their deployment and to a wider adoption of deep…

We describe a novel constructive technique for devising efficient first-order methods for a wide range of large-scale convex minimization settings, including smooth, non-smooth, and strongly convex minimization. The technique builds upon a…