Related papers: Non-normal spectral signatures of instability in n…
It has been recognized for quite some time that for some matrices the spectra are not enough to tell the complete story of the dynamics of the system, even for linear ODEs. While it is true that the eigenvalues control the asymptotic…
We develop a unified matrix-spectral framework for analyzing stability and interpretability in deep neural networks. Representing networks as data-dependent products of linear operators reveals spectral quantities governing sensitivity to…
The variation of spectral subspaces for linear self-adjoint operators under an additive bounded semidefinite perturbation is considered. A variant of the Davis-Kahan $ \sin2\Theta $ theorem from [SIAM J. Numer. Anal. 7 (1970), 1--46]…
Training modern neural networks is increasingly fragile, with rare but severe destabilizing updates often causing irreversible divergence or silent performance degradation. Existing optimization methods primarily rely on preventive…
In this research, we address critical concerns related to the numerical instability observed in 16-bit computations of machine learning models. Such instability, particularly when employing popular optimization algorithms like Adam, often…
Nonlinear dynamical systems can be handily described by the associated Koopman operator, whose action evolves every observable of the system forward in time. Learning the Koopman operator and its spectral decomposition from data is enabled…
Training instability in modern deep learning systems is frequently triggered by rare but extreme gradient-norm spikes, which can induce oversized parameter updates, corrupt optimizer state, and lead to slow recovery or divergence. Widely…
This paper investigates the stability properties of neural operators through the structured representation offered by the Hybrid B-spline Deep Neural Operator (HBDNO). While existing stability-aware architectures typically enforce…
This work establishes the first rigorous stability guarantees for approximate predictors in delay-adaptive control of nonlinear systems, addressing a key challenge in practical implementations where exact predictors are unavailable. We…
This paper is to study some conditions on semigroups, generated by some class of non-densely defined operators in the closure of its domain, in order that certain bounded perturbations preserve some regularity properties of the semigroup…
We investigate the effect of non-symmetric relatively bounded perturbations on the spectrum of self-adjoint operators. In particular, we establish stability theorems for one or infinitely many spectral gaps along with corresponding…
We consider the failure of localized control in a nonlinear spatially extended system caused by extremely small amounts of noise. It is shown that this failure occurs as a result of a nonlinear instability. Nonlinear instabilities can occur…
We bring together three key amplification mechanisms in linear dynamical systems: spectral criticality, resonance, and non-normality. We present a unified linear framework that both distinguishes and quantitatively links these effects…
Transformers trained in low precision can suffer forward-error amplification. We give a first-order, module-wise theory that predicts when and where errors grow. For self-attention we derive a per-layer bound that factorizes into three…
During neural network training, the sharpness of the Hessian matrix of the training loss rises until training is on the edge of stability. As a result, even nonstochastic gradient descent does not accurately model the underlying dynamical…
Computing pseudospectra of non-normal matrices is essential for understanding the stability and transient behavior of dynamical systems. Such analysis is critical in applications including fluid dynamics, control systems, and differential…
In this paper we consider Deep Neural Networks (DNNs) with a smooth activation function as surrogates for high-dimensional functions that are somewhat smooth but costly to evaluate. We consider the standard (non-periodic) DNNs as well as…
Koopman operator theory, a powerful framework for discovering the underlying dynamics of nonlinear dynamical systems, was recently shown to be intimately connected with neural network training. In this work, we take the first steps in…
Despite their impressive performance, contemporary neural networks often lack structural safeguards that promote stable learning and interpretable behavior. In this work, we introduce a reformulation of layer-level transformations that…
We propose a Regularized Adaptive Momentum Dual Averaging (RAMDA) algorithm for training structured neural networks. Similar to existing regularized adaptive methods, the subproblem for computing the update direction of RAMDA involves a…