Related papers: A Mechanism Study of Delayed Loss Spikes in Batch-…

Biological evidence suggests that adaptation of synaptic delays on short to medium timescales plays an important role in learning in the brain. Inspired by biology, we explore the feasibility and power of using synaptic delays to solve…

Batch Normalization (BatchNorm) is a widely adopted technique that enables faster and more stable training of deep neural networks (DNNs). Despite its pervasiveness, the exact reasons for BatchNorm's effectiveness are still poorly…

Over-parameterized models, such as large deep networks, often exhibit a double descent phenomenon, whereas a function of model size, error first decreases, increases, and decreases at last. This intriguing double descent behavior also…

Spiking Neural Networks (SNNs) have recently emerged as an alternative to deep learning owing to sparse, asynchronous and binary event (or spike) driven processing, that can yield huge energy efficiency benefits on neuromorphic hardware.…

We propose a deep unfolding-based approach for stabilization of time-delay linear systems. Deep unfolding is an emerging framework for design and improvement of iterative algorithms and attracting significant attentions in signal…

A fundamental problem in machine learning is understanding the effect of early stopping on the parameters obtained and the generalization capabilities of the model. Even for linear models, the effect is not fully understood for arbitrary…

Recent empirical and theoretical work has shown that the dynamics of the large eigenvalues of the training loss Hessian have some remarkably robust features across models and datasets in the full batch regime. There is often an early period…

Motivated by recent security issues in cyber-physical systems, this technical note studies the stabilization problem of networked control systems under Denial-of-Service (DoS) attacks. In particular, we consider to stabilize a nonlinear…

We investigate the stability of synchronized states in delay-coupled networks where synchronization takes place in groups of different local dynamics or in cluster states in networks with identical local dynamics. Using a master stability…

Learning how complex dynamical systems evolve over time is a key challenge in system identification. For safety critical systems, it is often crucial that the learned model is guaranteed to converge to some equilibrium point. To this end,…

Deep neural networks rely heavily on normalization methods to improve their performance and learning behavior. Although normalization methods spurred the development of increasingly deep and efficient architectures, they also increase the…

Discrete-time systems under aperiodic sampling may serve as a modeling abstraction for a multitude of problems arising in cyber-physical and networked control systems. Recently, model- and data-based stability conditions for such systems…

Motivated by packet routing in computer networks, online queuing systems are composed of queues receiving packets at different rates. Repeatedly, they send packets to servers, each of them treating only at most one packet at a time. In the…

Existing analyses of neural network training often operate under the unrealistic assumption of an extremely small learning rate. This lies in stark contrast to practical wisdom and empirical studies, such as the work of J. Cohen et al.…

The renormalization method which is a type of perturbation method is extended to a tool to study weakly nonlinear time-delay systems. For systems with order-one delay, we show that the renormalization method leads to reduced systems without…

We generalize a method of control of chaos which uses delayed feedback at the period of an unstable orbit to stabilize that orbit. The generalization consists of substituting some portion of the nonlinear dynamical system with a delayed…

Although overparameterized models have achieved remarkable practical success, their theoretical properties, particularly their generalization behavior, remain incompletely understood. The well known double descents phenomenon suggests that…

Deep neural networks exhibit periodic loss spikes during unregularized long-term training, a phenomenon known as the "Slingshot Mechanism." Existing work usually attributes this to intrinsic optimization dynamics, but its triggering…

Background: In neurophysiological data, latency refers to a global shift of spikes from one spike train to the next, either caused by response onset fluctuations or by finite propagation speed. Such systematic shifts in spike timing lead to…

This thesis addresses the question of stability of systems defined by differential equations which contain nonlinearity and delay. In particular, we analyze the stability of a well-known delayed nonlinear implementation of a certain…