Related papers: A Mechanism Study of Delayed Loss Spikes in Batch-…

In this paper a new distributed asynchronous algorithm is proposed for time synchronization in networks with random communication delays, measurement noise and communication dropouts. Three different types of the drift correction algorithm…

Spiking neural networks (SNNs) are rapidly gaining momentum as an alternative to conventional artificial neural networks in resource constrained edge systems. In this work, we continue a recent research line on recurrent SNNs where axonal…

Deep neural networks can achieve remarkable generalization performances while interpolating the training data perfectly. Rather than the U-curve emblematic of the bias-variance trade-off, their test error often follows a "double descent" -…

We investigate the combined effects of distributed delay and the balance between excitatory and inhibitory nodes on the stability of synchronous oscillations in a network of coupled Stuart--Landau oscillators. To this end a network model is…

This paper is a comprehensive study of a long observed phenomenon of increase in the stability margin and so the rate of convergence of a class of linear systems due to time delay. We use Lambert W function to determine (a) in what systems…

In this paper we develop novel results on self triggering control of nonlinear systems, subject to perturbations and actuation delays. First, considering an unperturbed nonlinear system with bounded actuation delays, we provide conditions…

This paper considers learning the hidden causal network of a linear networked dynamical system (NDS) from the time series data at some of its nodes -- partial observability. The dynamics of the NDS are driven by colored noise that generates…

The mean square stabilization problem for discrete-time networked control systems (NCSs) is investigated in this article. What the difference from most previous works is that input delay and packet losses occur simultaneously in the…

Over the past few years, Batch-Normalization has been commonly used in deep networks, allowing faster training and high performance for a wide variety of applications. However, the reasons behind its merits remained unanswered, with several…

In this work we investigate the reasons why Batch Normalization (BN) improves the generalization performance of deep networks. We argue that one major reason, distinguishing it from data-independent normalization methods, is randomness of…

We analyze the dynamics of networks of spiking neural oscillators. First, we present an exact linear stability theory of the synchronous state for networks of arbitrary connectivity. For general neuron rise functions, stability is…

This paper addresses the boundary output feedback stabilization of general 1-D reaction-diffusion PDEs with delayed boundary measurement. The output takes the form of a either Dirichlet or Neumann trace. The output delay can be arbitrarily…

This paper studies the problem of stability of a parameterized delay differential equations (DDE see equation (0.1)). After discretizing the DDE (0.1), we show that the problem can be equivalently casted into a semi-definite programming…

We obtain a result on the behavior of the solutions of a general nonautonomous Hopfield neural network model with delay, assuming some general bound for the product of consecutive terms in the sequence of neuron charging times and some…

Batch Normalization (BN) is a commonly used technique to accelerate and stabilize training of deep neural networks. Despite its empirical success, a full theoretical understanding of BN is yet to be developed. In this work, we analyze BN…

Pre-trained deep neural networks (DNNs) are being widely deployed by industry for making business decisions and to serve users; however, a major problem is model decay, where the DNN's predictions become more erroneous over time, resulting…

Spiking Neural Networks (SNNs) compute using sparse communication and are attracting increased attention as a more energy-efficient alternative to traditional Artificial Neural Networks~(ANNs). While standard ANNs are stateless, spiking…

Spike-timing-dependent plasticity (STDP) provides a biologically-plausible learning mechanism for spiking neural networks (SNNs); however, Hebbian weight updates in architectures with recurrent connections suffer from pathological weight…

Integral to recent successes in deep reinforcement learning has been a class of temporal difference methods that use infrequently updated target values for policy evaluation in a Markov Decision Process. Yet a complete theoretical…

When training deep neural networks with gradient descent, sharpness often increases -- a phenomenon known as progressive sharpening -- before saturating at the edge of stability. Although commonly observed in practice, the underlying…