Related papers: On the Stability Connection Between Discrete-Time …
We derive stability criteria for saddle points of a class of nonsmooth optimization problems in Hilbert spaces arising in PDE-constrained optimization, using metric regularity of infinite-dimensional set-valued mappings. A main ingredient…
We consider the problem of designing uniformly stable first-order optimization algorithms for empirical risk minimization. Uniform stability is often used to obtain generalization error bounds for optimization algorithms, and we are…
Accurate simulations of ice sheet dynamics, mantle convection, lava flow, and other highly viscous free-surface flows involve solving the coupled Stokes/free-surface equations. In this paper, we theoretically analyze the stability and…
There is an abundance of evidence that some relaxation dynamics, e.g., exponential decays, are much more common in nature than others. Recently, there have been attempts to trace this dominance back to a certain stability of the prevalent…
The Primal-Dual (PD) algorithm is widely used in convex optimization to determine saddle points. While the stability of the PD algorithm can be easily guaranteed, strict contraction is nontrivial to establish in most cases. This work…
The objective of this paper is to prove the convergence of a linear implicit multi-step numerical method for ordinary differential equations. The algorithm is obtained via Taylor approximations. The convergence is proved following the…
Many statistical estimators are defined as the fixed point of a data-dependent operator, with estimators based on minimizing a cost function being an important special case. The limiting performance of such estimators depends on the…
In this paper, we propose a dynamical systems perspective of the Expectation-Maximization (EM) algorithm. More precisely, we can analyze the EM algorithm as a nonlinear state-space dynamical system. The EM algorithm is widely adopted for…
We study the stability of general $n$-dimensional nonautonomous linear differential equations with infinite delays. Delay independent criteria, as well as criteria depending on the size of some finite delays are established. In the first…
Uniform stability is a notion of algorithmic stability that bounds the worst case change in the model output by the algorithm when a single data point in the dataset is replaced. An influential work of Hardt et al. (2016) provides strong…
Understanding how time delays impact the stability of a delay differential equation is important for modeling many natural and technological systems that experience time delays. Here we introduce a new stability criterion for…
First-order optimization algorithms can be considered as a discretization of ordinary differential equations (ODEs) \cite{su2014differential}. In this perspective, studying the properties of the corresponding trajectories may lead to…
Gradient descent and stochastic gradient descent are central to modern machine learning, yet their behavior under large step sizes remains theoretically unclear. Recent work suggests that acceleration often arises near the edge of…
We show that convex-concave Lipschitz stochastic saddle point problems (also known as stochastic minimax optimization) can be solved under the constraint of $(\epsilon,\delta)$-differential privacy with \emph{strong (primal-dual) gap} rate…
Explicit exponential stability tests are obtained for the scalar neutral differential equation $$ \dot{x}(t)-a(t)\dot{x}(g(t))=-\sum_{k=1}^m b_k(t)x(h_k(t)), $$ together with exponential estimates for its solutions. Estimates for solutions…
For a discrete dynamics defined by a sequence of bounded and not necessarily invertible linear operators, we give a complete characterization of exponential stability in terms of invertibility of a certain operator acting on suitable Banach…
Stiff systems of ordinary differential equations (ODEs) arise in a wide range of scientific and engineering disciplines and are traditionally solved using implicit integration methods due to their stability and efficiency. However, these…
We propose the first $\alpha$-parameterized framework for solving time-changed stochastic differential equations (TCSDEs), explicitly linking convergence rates to the driving parameter of the underlying stochastic processes. Theoretically,…
Minimax optimization problems have attracted a lot of attention over the past few years, with applications ranging from economics to machine learning. While advanced optimization methods exist for such problems, characterizing their…
Investigation of the approximation properties, convergence, and stability of the ADER-DG method for solving an ODE system is carried out. The ADER-DG method is $A$- and $AN$-stable, $L$-stable, $B$- and $BN$-stable, and algebraically…