Related papers: A Mechanism Study of Delayed Loss Spikes in Batch-…
This paper studies the problem of stabilization of a nonlinear system with time-varying delays in both sensing and actuation using event-triggered control. Our proposed strategy seeks to opportunistically minimize the number of control…
We study the possibility to stabilize unstable steady states and unstable periodic orbits in chaotic fractional-order dynamical systems by the time-delayed feedback method. By performing a linear stability analysis, we establish the…
Models with nonlinear architectures/parameterizations such as deep neural networks (DNNs) are well known for their mysteriously good generalization performance at overparameterization. In this work, we tackle this mystery from a novel…
In this paper, we exploit a diagonally dominant structure for the decentralized stabilization of unknown nonlinear time-delayed networks. To this end, we first introduce a novel generalization of virtual contraction analysis to diagonally…
The stability and generalization of stochastic gradient-based methods provide valuable insights into understanding the algorithmic performance of machine learning models. As the main workhorse for deep learning, stochastic gradient descent…
To model time series accurately is important within a wide range of fields. As the world is generally too complex to be modelled exactly, it is often meaningful to assess the probability of a dynamical system to be in a specific state. This…
Batch Normalization (BN) is capable of accelerating the training of deep models by centering and scaling activations within mini-batches. In this work, we propose Decorrelated Batch Normalization (DBN), which not just centers and scales…
We study delay-induced transitions in consensus dynamics on signed networks with a ring topology. The proposed model is formulated as a system of delay differential equations incorporating both cooperative and antagonistic interactions, as…
Normalization methods are a central building block in the deep learning toolbox. They accelerate and stabilize training, while decreasing the dependence on manually tuned learning rate schedules. When learning from multi-modal…
Grokking, the phenomenon of delayed generalization, is often attributed to the depth and compositional structure of deep neural networks. We study grokking in one of the simplest possible settings: the learning of a linear model with…
Precise timing of spikes and temporal locking are key elements of neural computation. Here we demonstrate how even strongly heterogeneous, deterministic neural networks with delayed interactions and complex topology can exhibit periodic…
Understanding the implicit bias of training algorithms is of crucial importance in order to explain the success of overparametrised neural networks. In this paper, we study the dynamics of stochastic gradient descent over diagonal linear…
Training state-of-the-art, deep neural networks is computationally expensive. One way to reduce the training time is to normalize the activities of the neurons. A recently introduced technique called batch normalization uses the…
Lifelong Model Editing aims to continuously update evolving facts in Large Language Models while preserving unrelated knowledge and general capabilities, yet it remains plagued by catastrophic forgetting and model collapse. Empirically, we…
The elapsed-time model describes the behavior of interconnected neurons through the time since their last spike. It is an age-structured non-linear equation in which age corresponds to the elapsed time since the last discharge, and models…
We present an event-triggered control strategy for stabilizing a scalar, continuous-time, time-invariant, linear system over a digital communication channel having bounded delay, and in the presence of bounded system disturbance. We propose…
In this article, we consider a nonlinear process with delayed dynamics to be controlled over a communication network in the presence of disturbances and study robustness of the resulting closed-loop system with respect to network-induced…
The optimization foundations of deep linear networks have recently received significant attention. However, due to their inherent non-convexity and hierarchical structure, analyzing the loss functions of deep linear networks remains a…
Normalization layers (e.g., Batch Normalization, Layer Normalization) were introduced to help with optimization difficulties in very deep nets, but they clearly also help generalization, even in not-so-deep nets. Motivated by the long-held…
In this paper, we consider the stability analysis of large-scale distributed networked control systems with random communication delays between linearly interconnected subsystems. The stability analysis is performed in the Markov jump…