Related papers: A Mechanism Study of Delayed Loss Spikes in Batch-…

The classical stability margin analysis based on the linearized model is widely used in practice even in nonlinear systems. Although linear analysis techniques are relatively standard and have simple implementation structures, they are…

In this paper we first study the fixed-time stabilizability of discrete-time switched linear control systems. Using a geometric approach, we derive conditions under which such systems can be stabilized within a prescribed number of steps,…

In overparameterized logistic regression, gradient descent (GD) iterates diverge in norm while converging in direction to the maximum $\ell_2$-margin solution -- a phenomenon known as the implicit bias of GD. This work investigates…

Denoising diffusion models have emerged as a dominant approach for image generation, however they still suffer from slow convergence in training and color shift issues in sampling. In this paper, we identify that these obstacles can be…

Among attempts at giving a theoretical account of the success of deep neural networks, a recent line of work has identified a so-called lazy training regime in which the network can be well approximated by its linearization around…

We study delay-independent stability in nonlinear models with a distributed delay which have a positive equilibrium. Such models frequently occur in population dynamics and other applications. In particular, we construct a relevant…

We present three examples of delayed bifurcations for spike solutions of reaction-diffusion systems. The delay effect results as the system passes slowly from a stable to an unstable regime, and was previously analysed in the context of…

When optimizing over-parameterized models, such as deep neural networks, a large set of parameters can achieve zero training error. In such cases, the choice of the optimization algorithm and its respective hyper-parameters introduces…

We develop a stabilization strategy of turning processes by means of delayed spindle control. We show that turning processes which contain intrinsic state-dependent delays can be stabilized by a spindle control with state-dependent delay,…

In the experiments on stress-induced phase transitions in SMA strips, several interesting instability phenomena have been observed, including a necking-type instability, a shear-type instability and an orientation instability. By using the…

Neural-network processing in machine learning applications relies on layer synchronization. This is practiced even in artificial Spiking Neural Networks (SNNs), which are touted as consistent with neurobiology, in spite of processing in the…

To date, the instability of prognostic predictors in a sparse high dimensional model, which hinders their clinical adoption, has received little attention. Stable prediction is often overlooked in favour of performance. Yet, stability…

Working memory -- the ability to store and recall precise temporal patterns of neural activity -- remains an open challenge for spiking neural networks (SNNs). We propose a recurrent SNN of $N$ neurons in which each synapse is equipped with…

Biological neural networks face a formidable task: performing reliable computations in the face of intrinsic stochasticity in individual neurons, imprecisely specified synaptic connectivity, and nonnegligible delays in synaptic…

Background: Deep learning models are typically trained using stochastic gradient descent or one of its variants. These methods update the weights using their gradient, estimated from a small fraction of the training data. It has been…

Stability arguments are often used to prevent learning algorithms from having ever increasing activity and weights that hinder generalization. However, stability conditions can clash with the sparsity required to augment the energy…

This work studies the mean-square stability and stabilization problem for networked feedback systems. Data transmission delays in the network channels of the systems are considered. It is assumed that these delays are i.i.d. processes with…

Batch normalization (batch norm) is often used in an attempt to stabilize and accelerate training in deep neural networks. In many cases it indeed decreases the number of parameter updates required to achieve low training error. However, it…

Randomly initialized neural networks are known to become harder to train with increasing depth, unless architectural enhancements like residual connections and batch normalization are used. We here investigate this phenomenon by revisiting…

Most existing literature on supply chain and inventory management consider stochastic demand processes with zero or constant lead times. While it is true that in certain niche scenarios, uncertainty in lead times can be ignored, most…