Related papers: Input-Output Stability of Gradient Descent: A Disc…

Stochastic gradient descent is a classic algorithm that has gained great popularity especially in the last decades as the most common approach for training models in machine learning. While the algorithm has been well-studied when…

Gradient Descent Ascent (GDA) methods for min-max optimization problems typically produce oscillatory behavior that can lead to instability, e.g., in bilinear settings. To address this problem, we introduce a dissipation term into the GDA…

This work establishes new convergence guarantees for gradient descent in smooth convex optimization via a computer-assisted analysis technique. Our theory allows nonconstant stepsize policies with frequent long steps potentially violating…

Many differential equations with physical backgrounds are described as gradient systems, which are evolution equations driven by the gradient of some functionals, and such problems have energy conservation or dissipation properties. For…

We develop new sub-optimality bounds for gradient descent (GD) that depend on the conditioning of the objective along the path of optimization rather than on global, worst-case constants. Key to our proofs is directional smoothness, a…

We introduce $\mathbf{G}$radient Descent with $\mathbf{A}$daptive $\mathbf{M}$omentum $\mathbf{S}$caling ($\mathbf{Grams}$), a novel optimization algorithm that decouples the direction and magnitude of parameter updates in deep learning.…

Model-free reinforcement learning attempts to find an optimal control action for an unknown dynamical system by directly searching over the parameter space of controllers. The convergence behavior and statistical properties of these…

Efficient approaches to quantum control and feedback are essential for quantum technologies, from sensing to quantum computation. Open-loop control tasks have been successfully solved using optimization techniques, including methods like…

One approach for feedback control using high dimensional and rich sensor measurements is to classify the measurement into one out of a finite set of situations, each situation corresponding to a (known) control action. This approach…

It seems that in the current age, computers, computation, and data have an increasingly important role to play in scientific research and discovery. This is reflected in part by the rise of machine learning and artificial intelligence,…

This paper presents an extremum seeking control algorithm with an adaptive step-size that adjusts the aggressiveness of the controller based on the quality of the gradient estimate. The adaptive step-size ensures that the integral-action…

The stochastic gradient descent (SGD) method is a widely used approach for solving stochastic optimization problems, but its convergence is typically slow. Existing variance reduction techniques, such as SAGA, improve convergence by…

We address the challenge of estimating the learning rate for adaptive gradient methods used in training deep neural networks. While several learning-rate-free approaches have been proposed, they are typically tailored for steepest descent.…

In the paper https://doi.org/10.1016/j.physleta.2011.08.072 authors propose a modification of the conventional delayed feedback control algorithm, where time-delay is varied continuously to minimize the power of control force. Minimization…

A novel method for control of dynamical systems, proposed in the paper, ensures an output signal belonging to the given set at any time. The method is based on a special change of coordinates such that the initial problem with given…

We consider the static output feedback control for Linear Quadratic Regulator problems with structured constraints under the assumption that system parameters are unknown. To solve the problem in the model free setting, we propose the…

We show that gradient descent converges to a local minimizer, almost surely with random initialization. This is proved by applying the Stable Manifold Theorem from dynamical systems theory.

This manuscript introduces a passivity-based integral control approach for fully-actuated mechanical systems. The novelty of our methodology is that we exploit the gyroscopic forces of the mechanical systems to exponentially stabilize the…

Output feedback stabilization of control systems is a crucial issue in engineering. Most of these systems are not uniformly observable, which proves to be a difficulty to move from state feedback stabilization to dynamic output feedback…

Dynamic surface control (DSC) method uses high gain filters to avoid the "explosion of complexity" issue inherent in backstepping based controller designs. As a result, the closed loop system and filter dynamics possess time scale…