Related papers: Gradient Clock Synchronization with Practically Co…
A clock synchronizing circuit for repeaterless low swing interconnects is presented in this paper. The circuit uses a delay locked loop (DLL) to generate multiple phases of the clock, of which the one closest to the center of the eye is…
Synchronization problems of continuous and discrete singularly perturbed systems are studied in this paper with singular perturbations and time scales (SPaTS) technique. The dynamics of leader and followers are decomposed into pure-slow and…
We study a generic class of decentralized algorithms in which $N$ agents jointly optimize the non-convex objective $f(u):=1/N\sum_{i=1}^{N}f_i(u)$, while only communicating with their neighbors. This class of problems has become popular in…
A major challenge faced in the design of large-scale cyber-physical systems, such as power systems, the Internet of Things or intelligent transportation systems, is that traditional distributed optimal control methods do not scale…
Gradient clipping is a popular modification to standard (stochastic) gradient descent, at every iteration limiting the gradient norm to a certain value $c >0$. It is widely used for example for stabilizing the training of deep learning…
The phenomenon that stochastic gradient descent (SGD) favors flat minima has played a critical role in understanding the implicit regularization of SGD. In this paper, we provide an explanation of this striking phenomenon by relating the…
We report the nature of transitions from nonsynchronous to complete synchronization (CS) state in arrays of time-delay systems, where the systems are coupled with instantaneous diffusive coupling. We demonstrate that the transition to CS…
In-phase synchronization is a special case of synchronous behavior when coupled oscillators have the same phases for any time moments. Such behavior appears naturally for nearly identical coupled limit-cycle oscillators when the coupling…
Forming (hybrid) AC/DC microgrids (MGs) has become a promising manner for the interconnection of various kinds of distributed generators that are inherently AC or DC electric sources. This paper addresses the distributed asynchronous power…
Stochastic Gradient Descent (SGD) is the standard numerical method used to solve the core optimization problem for the vast majority of machine learning (ML) algorithms. In the context of large scale learning, as utilized by many Big Data…
In this paper, we consider the synchronization of heterogeneous pulse-coupled oscillators (PCOs), where some of the oscillators might be faulty or malicious. The oscillators interact through identical pulses at discrete instants and evolve…
Local SGD is a popular optimization method in distributed learning, often outperforming other algorithms in practice, including mini-batch SGD. Despite this success, theoretically proving the dominance of local SGD in settings with…
Graph Convolutional Networks (GCNs) show promising results for semi-supervised learning tasks on graphs, thus become favorable comparing with other approaches. Despite the remarkable success of GCNs, it is difficult to train GCNs with…
Stochastic gradient descent (SGD) is a simple and popular method to solve stochastic optimization problems which arise in machine learning. For strongly convex problems, its convergence rate was known to be O(\log(T)/T), by running SGD for…
Sketched gradient algorithms have been recently introduced for efficiently solving the large-scale constrained Least-squares regressions. In this paper we provide novel convergence analysis for the basic method {\it Gradient Projection…
Recent research has observed that in machine learning optimization, gradient descent (GD) often operates at the edge of stability (EoS) [Cohen, et al., 2021], where the stepsizes are set to be large, resulting in non-monotonic losses…
Large-scale integration of inverter-based resources into power grids worldwide is challenging their stability and security. This paper takes a closer look at synchronous condensers as a solution to mitigate stability challenges caused by…
This paper proposes a thorough theoretical analysis of Stochastic Gradient Descent (SGD) with non-increasing step sizes. First, we show that the recursion defining SGD can be provably approximated by solutions of a time inhomogeneous…
Recently, the technique of local updates is a powerful tool in centralized settings to improve communication efficiency via periodical communication. For decentralized settings, it is still unclear how to efficiently combine local updates…
Virtualization of distributed real-time systems enables the consolidation of mixed-criticality functions on a shared hardware platform thus easing system integration. Time-triggered communication and computation can act as an enabler of…