Related papers: Stable non-minimal fixed points of threshold-linea…
Monotonicity constraints are powerful regularizers in statistical modelling. They can support fairness in computer-aided decision making and increase plausibility in data-driven scientific models. The seminal min-max (MM) neural network…
In this paper, we introduce the notion of liquid time-constant (LTC) recurrent neural networks (RNN)s, a subclass of continuous-time RNNs, with varying neuronal time-constant realized by their nonlinear synaptic transmission model. This…
We study the stable behaviour of discrete dynamical systems where the map is convex and monotone with respect to the standard positive cone. The notion of tangential stability for fixed points and periodic points is introduced, which is…
This paper investigates asymptotic fixed point results for nonlinear contractions, with emphasis on Kirk-type theorems and their generalizations. A central difficulty in the literature has been the requirement that the mapping possesses a…
We show that for any positive integer $d$, there are families of switched linear systems---in fixed dimension and defined by two matrices only---that are stable under arbitrary switching but do not admit (i) a polynomial Lyapunov function…
In this work, we study the nonlinear dynamics of a shallow neural network trained with mean-squared loss and leaky ReLU activation. Under Gaussian inputs and equal layer width k, (1) we establish, based on the equivariant gradient degree, a…
Learning control strategies with provable stability guarantees continues to be a challenging problem. In this work, we examine a family of training-time behaviors exhibited by existing neural Lyapunov control methods under specific…
We consider planar networks of three curves that meet at two junctions with prescribed equal angles, minimizing a combination of the elastic energy and the length functional. We prove existence and regularity of minimizers, and we show some…
Structural changes occur in dynamic networks quite frequently and its detection is an important question in many situations such as fraud detection or cybersecurity. Real-life networks are often incompletely observed due to individual…
Finding minimum distortion of adversarial examples and thus certifying robustness in neural network classifiers for given data points is known to be a challenging problem. Nevertheless, recently it has been shown to be possible to give a…
We consider a minimizing variant of the well-known \emph{No-Three-In-Line Problem}, the \emph{Geometric Dominating Set Problem}: What is the smallest number of points in an $n\times n$~grid such that every grid point lies on a common line…
This paper presents a theoretical analysis of linear interpolation as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity…
We identify stable regions in the residual stream of Transformers, where the model's output remains insensitive to small activation changes, but exhibits high sensitivity at region boundaries. These regions emerge during training and become…
This paper deals with transient stability in interconnected micro-grids. The main contribution involves i) robust classification of transient dynamics for different intervals of the micro-grid parameters (synchronization, inertia, and…
We use smoothed analysis techniques to provide guarantees on the training loss of Multilayer Neural Networks (MNNs) at differentiable local minima. Specifically, we examine MNNs with piecewise linear activation functions, quadratic loss and…
Fixed-point optimization of deep neural networks plays an important role in hardware based design and low-power implementations. Many deep neural networks show fairly good performance even with 2- or 3-bit precision when quantized weights…
Modern neural networks are often quite wide, causing large memory and computation costs. It is thus of great interest to train a narrower network. However, training narrow neural nets remains a challenging task. We ask two theoretical…
We give a survey, known and new results on the beingness of fixed points of the maximal operator in the more general settings of metric measure space. In particular, we prove that the fixed points of the uncentered one must be the constant…
It is well known that (stochastic) gradient descent has an implicit bias towards flat minima. In deep neural network training, this mechanism serves to screen out minima. However, the precise effect that this has on the trained network is…
This paper discusses a general and useful stability principle which, roughly speaking, says that given a uniformly continuous function defined on an arbitrary metric space, if the function is bounded on the constraint set and we slightly…