Related papers: Gradient- and Newton-Based Unit Vector Extremum Se…
This paper introduces extremum seeking (ES) algorithms designed to achieve perfect tracking of arbitrary time-varying extremum. In contrast to classical ES approaches that employ constant frequencies and controller gains, our algorithms…
This paper proposes a multivariable extremum seeking scheme using Fast Fourier Transform (FFT) for a network of subsystems working towards optimizing the sum of their local objectives, where the overall objective is the only available…
The stochastic gradient descent (SGD) method is a widely used approach for solving stochastic optimization problems, but its convergence is typically slow. Existing variance reduction techniques, such as SAGA, improve convergence by…
Stochastic coordinate descent algorithms are efficient methods in which each iterate is obtained by fixing most coordinates at their values from the current iteration, and approximately minimizing the objective with respect to the remaining…
Convergence detection of iterative stochastic optimization methods is of great practical interest. This paper considers stochastic gradient descent (SGD) with a constant learning rate and momentum. We show that there exists a transient…
Variational approaches to disparity estimation typically use a linearised brightness constancy constraint, which only applies in smooth regions and over small distances. Accordingly, current variational approaches rely on a schedule to…
This paper reveals that a common and central role, played in many error bound (EB) conditions and a variety of gradient-type methods, is a residual measure operator. On one hand, by linking this operator with other optimality measures, we…
In recent years, accelerated extra-gradient methods have attracted much attention by researchers, for solving monotone inclusion problems. A limitation of most current accelerated extra-gradient methods lies in their direct utilization of…
We study the trade-offs between convergence rate and robustness to gradient errors in designing a first-order algorithm. We focus on gradient descent (GD) and accelerated gradient (AG) methods for minimizing strongly convex functions when…
This paper presents an algorithmic framework for solving unconstrained stochastic optimization problems using only stochastic function evaluations. We employ central finite-difference based gradient estimation methods to approximate the…
In this paper, acceleration of gradient methods for convex optimization problems with weak levels of convexity and smoothness is considered. Starting from the universal fast gradient method which was designed to be an optimal method for…
In this paper, a novel stochastic extra-step quasi-Newton method is developed to solve a class of nonsmooth nonconvex composite optimization problems. We assume that the gradient of the smooth part of the objective function can only be…
A broad class of hybrid quantum-classical algorithms known as "variational algorithms" have been proposed in the context of quantum simulation, machine learning, and combinatorial optimization as a means of potentially achieving a quantum…
In this paper, we study the control of dynamical systems under temporal logic task specifications using gradient-based methods relying on quantitative measures that express the extent to which the tasks are satisfied. A class of controllers…
A number of optimization approaches have been proposed for optimizing nonconvex objectives (e.g. deep learning models), such as batch gradient descent, stochastic gradient descent and stochastic variance reduced gradient descent. Theory…
Designing effective optimisation strategies for unsteady flows in the presence of complex dynamics is challenging. Gradient-based optimisation algorithms that rely on gradient information obtained from adjoint equations are efficient for…
A scaled conjugate gradient method that accelerates existing adaptive methods utilizing stochastic gradients is proposed for solving nonconvex optimization problems with deep neural networks. It is shown theoretically that, whether with…
In [22] a form of extremum seeking for control (ESC) was developed for the stabilization of uncertain nonlinear systems. In ESC the extremum seeker itself controls the systems through feedback rather than fine tuning a controller. The ESC…
Extremum-seeking control (ESC) is widely used to optimize performance when the system dynamics are uncertain. However, sensitivity to sensor noise is a crucial issue in ESC implementation due to the use of high-pass filters or gradient…
One common approach to detecting change-points is minimizing a cost function over possible numbers and locations of change-points. The framework includes several well-established procedures, such as the penalized likelihood and minimum…