Related papers: Gradient- and Newton-Based Unit Vector Extremum Se…
There have been recent efforts that combine seemingly disparate methods, extremum seeking (ES) optimization and partial differential equation (PDE) backstepping, to address the problem of model-free optimization with PDE actuator dynamics.…
In this paper, we develop and analyze an integral fixed-time sliding mode control method for a scenario in which the system model is only partially known, utilizing Gaussian processes. We present two theorems on fixed-time convergence. The…
Treating optimization methods as dynamical systems can be traced back centuries ago in order to comprehend the notions and behaviors of optimization methods. Lately, this mind set has become the driving force to design new optimization…
This paper focuses on the further development of the Lie bracket approximation approach for extremum seeking systems. Classical results in this area provide extremum seeking algorithms with exponential convergence rates for quadratic-like…
This paper can be seen as an attempt of rethinking the {\em Extra-Gradient Philosophy} for solving Variational Inequality Problems. We show that the properly defined {\em Reduced Gradients} can be used instead for finding approximate…
This paper deals with the gradient-based extremum seeking control for multivariable maps under actuator saturation. By exploiting a polytopic embedding of the unknown Hessian, we derive a LMI-based synthesis condition to ensure that the…
Stochastic Gradient Descent-Ascent (SGDA) is one of the most prominent algorithms for solving min-max optimization and variational inequalities problems (VIP) appearing in various machine learning tasks. The success of the method led to…
In this paper, a class of nonlinear driftless control-affine systems satisfying the bracket generating condition is considered. A gradient-free optimization algorithm is developed for the minimization of a cost function along the…
Sign Gradient Descent (SignGD) is a simple yet robust optimization method, widely used in machine learning for its resilience to gradient noise and compatibility with low-precision computations. While its empirical performance is well…
This paper proposes novel gradient-flow schemes that yield convergence to the optimal point of a convex optimization problem within a \textit{fixed} time from any given initial condition for unconstrained optimization, constrained…
In this paper, we study the gradient descent-ascent method for convex-concave saddle-point problems. We derive a new non-asymptotic global convergence rate in terms of distance to the solution set by using the semidefinite programming…
Variational empirical Bayes (VEB) methods provide a practically attractive approach to fitting large, sparse, multiple regression models. These methods usually use coordinate ascent to optimize the variational objective function, an…
A second-order Newton-based extremum seeking (SONES) algorithm is presented to estimate directional inflection points for multivariable static maps. The design extends the first-order Newton-based extremum seeking algorithm that drives the…
Our approach is part of the close link between continuous dissipative dynamical systems and optimization algorithms. We aim to solve convex minimization problems by means of stochastic inertial differential equations which are driven by the…
Machine learning models are often learned by minimising a loss function on the training data using a gradient descent algorithm. These models often suffer from overfitting, leading to a decline in predictive performance on unseen data. A…
This paper studies a stochastic extremum seeking method to steer a nonholonomic vehicle to the unknown source of a static spatially distributed filed in a plane. The key challenge lies in the lack of vehicle's position information and the…
In this paper, we describe a broad class of control functions for extremum seeking problems. We show that it unifies and generalizes existing extremum seeking strategies which are based on Lie bracket approximations, and allows to design…
We propose AEGD, a new algorithm for first-order gradient-based optimization of non-convex objective functions, based on a dynamically updated energy variable. The method is shown to be unconditionally energy stable, irrespective of the…
Stochastic gradient descent (SGD), which dates back to the 1950s, is one of the most popular and effective approaches for performing stochastic optimization. Research on SGD resurged recently in machine learning for optimizing convex loss…
We propose a gradient-based method for quadratic programming problems with a single linear constraint and bounds on the variables. Inspired by the GPCG algorithm for bound-constrained convex quadratic programming [J.J. Mor\'e and G.…