Related papers: Extremum seeking with exponential convergence via …
The paper deals with a well-known extremum seeking scheme by proving uniformity properties with respect to the amplitudes of the dither signal and of the cost function. Those properties are then used to show that the scheme guarantees the…
Many classical and modern machine learning algorithms require solving optimization tasks under orthogonality constraints. Solving these tasks with feasible methods requires a gradient descent update followed by a retraction operation on the…
We present multivariable extremum seeking (ES) designs that achieve unbiased convergence to the optimum. Two designs are introduced: one with exponential unbiased convergence (unbiased extremum seeker, uES) and the other with…
Zeroth-order methods are extensively used in machine learning applications where gradients are infeasible or expensive to compute, such as black-box attacks, reinforcement learning, and language model fine-tuning. Existing optimization…
This paper considers decentralized consensus optimization problems where different summands of a global objective function are available at nodes of a network that can communicate with neighbors only. The proximal method of multipliers is…
In this paper, we propose an inexact proximal Newton-type method for nonconvex composite problems. We establish the global convergence rate of the order $\mathcal{O}(k^{-1/2})$ in terms of the minimal norm of the KKT residual mapping and…
In minimum-cost inverse optimization problems, we are given a feasible solution to an underlying optimization problem together with a linear cost function, and the goal is to modify the costs by a small deviation vector so that the input…
A recently introduced general-purpose heuristic for finding high-quality solutions for many hard optimization problems is reviewed. The method is inspired by recent progress in understanding far-from-equilibrium phenomena in terms of {\em…
A second-order Newton-based extremum seeking (SONES) algorithm is presented to estimate directional inflection points for multivariable static maps. The design extends the first-order Newton-based extremum seeking algorithm that drives the…
In this manuscript, we address continuous unconstrained multi-objective optimization problems and we discuss descent type methods for the reconstruction of the Pareto set. Specifically, we analyze the class of Front Descent methods, which…
In this work, we investigate an efficient numerical approach for solving higher order statistical methods for blind and semi-blind signal recovery from non-ideal channels. We develop numerical algorithms based on convex optimization…
This work proposes and analyzes a compressed sensing approach to polynomial approximation of complex-valued functions in high dimensions. Of particular interest is the setting where the target function is smooth, characterized by a rapidly…
The minimisation of cost functions is crucial in various optimisation fields. However, identifying their global minimum remains challenging owing to the huge computational cost incurred. This work analytically expresses the computational…
Second-order methods for neural network optimization have several advantages over methods based on first-order gradient descent, including better scaling to large mini-batch sizes and fewer updates needed for convergence. But they are…
This paper proposes a multivariable extremum seeking scheme using Fast Fourier Transform (FFT) for a network of subsystems working towards optimizing the sum of their local objectives, where the overall objective is the only available…
We introduce a search problem generalizing the typical setting of Binary Search on the line. Similar to the setting for Binary Search, a target is chosen adversarially on the line, and in response to a query, the algorithm learns whether…
This paper studies stochastic minimization of a finite-sum loss $ F (\mathbf{x}) = \frac{1}{N} \sum_{\xi=1}^N f(\mathbf{x};\xi) $. In many real-world scenarios, the Hessian matrix of such objectives exhibits a low-rank structure on a batch…
This paper proposes near-optimal algorithms for the pure-exploration linear bandit problem in the fixed confidence and fixed budget settings. Leveraging ideas from the theory of suprema of empirical processes, we provide an algorithm whose…
Control-affine Extremum Seeking Control (ESC) systems have been increasingly studied and applied in the last decade. In a recent effort, many control-affine ESC structures have been generalized in a unifying class and their stability was…
This paper can be seen as an attempt of rethinking the {\em Extra-Gradient Philosophy} for solving Variational Inequality Problems. We show that the properly defined {\em Reduced Gradients} can be used instead for finding approximate…