Related papers: A Mechanism Study of Delayed Loss Spikes in Batch-…

We analyze the stabilization of unstable steady states by delayed feedback control with a periodic time-varying delay in the regime of a high-frequency modulation of the delay. The average effect of the delayed feedback term in the control…

The dynamical stability of the iterates during training plays a key role in determining the minima obtained by optimization algorithms. For example, stable solutions of gradient descent (GD) correspond to flat minima, which have been…

We consider the problem of adaptive stabilization for discrete-time, multi-dimensional linear systems with bounded control input constraints and unbounded stochastic disturbances, where the parameters of the true system are unknown. To…

Diffusion models generate structure by progressively transforming noise into data, yet the mechanisms underlying this transition remain poorly understood. In this work, we show that pattern formation in trained diffusion models can be…

A dynamical system is said to undergo rate-induced tipping when it fails to track its quasi-equilibrium state due to an above-critical-rate change of system parameters. We study a prototypical model for rate-induced tipping, the saddle-node…

Deep neural networks (DNNs) are typically optimized using various forms of mini-batch gradient descent algorithm. A major motivation for mini-batch gradient descent is that with a suitably chosen batch size, available computing resources…

Why do neural networks trained with large learning rates for a longer time often lead to better generalization? In this paper, we delve into this question by examining the relation between training and testing loss in neural networks.…

Notable progress has been made in numerous fields of machine learning based on neural network-driven mutual information (MI) bounds. However, utilizing the conventional MI-based losses is often challenging due to their practical and…

Deep learning has achieved remarkable success across a wide range of tasks, but its models often suffer from instability and vulnerability: small changes to the input may drastically affect predictions, while optimization can be hindered by…

Spiking Neural Networks (SNNs) are dynamical systems that operate on spatiotemporal data, yet their learnable parameters are often limited to synaptic weights, contributing little to temporal pattern recognition. Learnable parameters that…

In this paper, we introduce a novel approach to solve the (mean-covariance) steering problem for a fairly general class of linear continuous-time stochastic systems subject to input delays. Specifically, we aim at steering delayed linear…

We consider shallow (single hidden layer) neural networks and characterize their performance when trained with stochastic gradient descent as the number of hidden units $N$ and gradient descent steps grow to infinity. In particular, we…

This paper studies the event-triggered control problem for time-delay systems. A novel event-triggering scheme is proposed to exponentially stabilize a class of linear time-delay systems. By employing a new Halanay-type inequality and the…

We develop a predictor-feedback control design for multi-input nonlinear systems with distinct input delays, of arbitrary length, in each individual input channel. Due to the fact that different input signals reach the plant at different…

Deep Neural Networks (DNNs) have begun to thrive in the field of automation systems, owing to the recent advancements in standardising various aspects such as architecture, optimization techniques, and regularization. In this paper, we take…

Spiking Neural Networks (SNNs) are widely regarded as an energy-efficient paradigm for modeling and processing temporal and event-driven information. Incorporating delays in SNNs has been proven to be an effective mechanism for improving…

We show that a cumulative action of noise and delayed feedback on an excitable theta-neuron leads to rather coherent stochastic bursting. An idealized point process, valid if the characteristic time scales in the problem are well-separated,…

In this paper, we first present an explanation regarding the common occurrence of spikes in the training loss when neural networks are trained with stochastic gradient descent (SGD). We provide evidence that the spikes in the training loss…

We consider the deterministic evolution of a time-discretized spiking network of neurons with connection weights having delays, modeled as a discretized neural network of the generalized integrate and fire (gIF) type. The purpose is to…

In this paper, we consider a stabilization problem of an uncertain system in a networked control setting. Due to the network, the measurements are quantized to finite-bit signals and may be randomly lost in the communication. We study…