Related papers: Perturbed Gradient Descent Algorithms are Small-Di…
This paper deals with input/output-to-state stability (IOSS) of switched nonlinear systems in the discrete-time setting. We present an algorithm to construct periodic switching signals that obey pre-specified restrictions on admissible…
Stochastic gradient descent is a classic algorithm that has gained great popularity especially in the last decades as the most common approach for training models in machine learning. While the algorithm has been well-studied when…
The small gain condition is sufficient for input-to-state stability (ISS) of interconnected systems. However, verification of the small gain condition requires large amount of computations in the case of a large size of the system. To…
The article discusses distributed gradient-descent algorithms for computing local and global minima in nonconvex optimization. For local optimization, we focus on distributed stochastic gradient descent (D-SGD)--a simple network-based…
This paper addresses characterizations of integral input-to-state stability (iISS) for hybrid systems. In particular, we give a Lyapunov characterization of iISS unifying and generalizing the existing theory for pure continuous-time and…
In this paper, we study the implicit regularization of stochastic gradient descent (SGD) through the lens of {\em dynamical stability} (Wu et al., 2018). We start by revising existing stability analyses of SGD, showing how the Frobenius…
Low-rank matrix estimation is a canonical problem that finds numerous applications in signal processing, machine learning and imaging science. A popular approach in practice is to factorize the matrix into two compact low-rank factors, and…
We introduce the concept of non-uniform input-to-state stability for networks. It combines the uniform global stability with the uniform attractivity of any subnetwork, while it allows for non-uniform convergence of all components. For an…
Stochastic gradient descent (SGD) is a popular algorithm for minimizing objective functions that arise in machine learning. For constant step-sized SGD, the iterates form a Markov chain on a general state space. Focusing on a class of…
Modern machine learning algorithms aim to extract fine-grained information from data to provide accurate predictions, which often conflicts with the goal of privacy protection. This paper addresses the practical and theoretical importance…
The success of deep learning is due, to a large extent, to the remarkable effectiveness of gradient-based optimization methods applied to large neural networks. The purpose of this work is to propose a modern view and a general mathematical…
The dynamical stability of the iterates during training plays a key role in determining the minima obtained by optimization algorithms. For example, stable solutions of gradient descent (GD) correspond to flat minima, which have been…
Large-scale interconnected networks, composed of multiple low-dimensional subsystems, serve as a crucial framework for modeling a wide range of real-world applications. Despite offering computational scalability, the inherent…
This paper addresses the steady state covariance steering for linear dynamical systems via structural intervention on the system matrix. We formulate the covariance steering problem as the minimization of the Kullback-Leibler (KL)…
This paper is devoted to two issues. One is to provide Lyapunov-based tools to establish integral input-to-state stability (iISS) and input-to-state stability (ISS) for some classes of nonlinear parabolic equations. The other is to provide…
This paper presents a maximum principle-based approach in the establishment of input-to-state stability (ISS) for a class of nonlinear parabolic partial differential equations (PDEs) over higher dimensional domains with variable…
We study the input-to-state stability (ISS) of boundary control systems allowing for infinitely many boundary couplings. Using semigroup perturbation theory and the theory of positive linear operators on Banach lattices, we derive a…
This work primarily focuses on synthesizing a controller that guarantees an unknown continuous-time system to be incrementally input-to-state stable ($\delta$-ISS). In this context, the notion of $\delta$-ISS control Lyapunov function…
Input-to-state stability (ISS) of switched systems is studied where the individual subsystems are connected in a serial cascade configuration, and the states are allowed to reset at switching times. An ISS Lyapunov function is associated to…
Over the past decade, stochastic algorithms have emerged as scalable and efficient tools for solving large-scale ill-posed inverse problems by randomly selecting subsets of equations at each iteration. However, due to the ill-posedness and…