Related papers: Input-Output Stability of Gradient Descent: A Disc…
Stochastic gradient descent (SGD) has been widely used in machine learning due to its computational efficiency and favorable generalization properties. Recently, it has been empirically demonstrated that the gradient noise in several deep…
We investigate the test risk of continuous-time stochastic gradient flow dynamics in learning theory. Using a path integral formulation we provide, in the regime of a small learning rate, a general formula for computing the difference…
Gradient-based iterative optimization methods are the workhorse of modern machine learning. They crucially rely on careful tuning of parameters like learning rate and momentum. However, one typically sets them using heuristic approaches…
Transformer models have emerged as fundamental tools across various scientific and engineering disciplines, owing to their outstanding performance in diverse applications. Despite this empirical success, the theoretical foundations of…
We study the gradient method under the assumption that an additively inexact gradient is available for, generally speaking, non-convex problems. The non-convexity of the objective function, as well as the use of an inexactness specified…
Recently developed control methods with strong disturbance rejection capabilities provide a useful option for control design. The key lies in a general concept of disturbance and effective ways to estimate and compensate the disturbance.…
Imitation learning enables the synthesis of controllers for complex objectives and highly uncertain plant models. However, methods to provide stability guarantees to imitation learned controllers often rely on large amounts of data and/or…
In this paper, we present a novel control scheme for feedback optimization. That is, we propose a discrete-time controller that can steer the steady state of a physical plant to the solution of a constrained optimization problem without…
In this paper we explore the stabilization of closed invariant sets for passive systems, and present conditions under which a passivity-based feedback asymptotically stabilizes the goal set. Our results rely on novel reduction principles…
Due to their relevance in controller design, we consider the problem of determining the $\mathcal{L}^2$-gain, passivity properties and conic relations of an input-output system. While, in practice, the input-output relation is often…
In the present work, sufficient conditions for global stabilization of nonlinear uncertain systems by means of discrete-delay static output feedback are presented. Illustrating examples show the efficiency of the proposed control strategy.
We study a fixed step-size noisy distributed gradient descent algorithm for solving optimization problems in which the objective is a finite sum of smooth but possibly non-convex functions. Random perturbations are introduced to the…
The key idea behind PID Passivity-based Control (PID-PBC) is to leverage the passivity property of PIDs (for all positive gains) and wrap the PID controller around a passive output to ensure global stability in closed-loop. However, the…
Optimization problems with continuous data appear in, e.g., robust machine learning, functional data analysis, and variational inference. Here, the target function is given as an integral over a family of (continuously) indexed target…
This paper proposes a generalized passivity sensitivity analysis for power system stability studies. The method uncovers the most effective instability mitigation actions for both device-level and system-level investigations. The particular…
It is known that input-output approaches based on scaled small-gain theorems with constant $D$-scalings and integral linear constraints are non-conservative for the analysis of some classes of linear positive systems interconnected with…
We consider the classical gradient descent algorithm with constant stepsizes, where some error is introduced in the computation of each gradient. More specifically, we assume some relative bound on the inexactness, in the sense that the…
This paper studies static output feedback stabilization of continuous-time (incrementally) passive nonlinear systems where the control actions can only be chosen from a discrete (and possibly finite) set of points. For this purpose, we are…
We consider the problem of zeroing an error output of a nonlinear discrete-time system in the presence of constant exogenous disturbances, subject to hard convex constraints on the input signal. The design specification is formulated as a…
Recent studies show that transformer-based architectures emulate gradient descent during a forward pass, contributing to in-context learning capabilities - an ability where the model adapts to new tasks based on a sequence of prompt…