Related papers: Distributed Phasers
In a network of dynamical systems, concurrent synchronization is a regime where multiple groups of fully synchronized elements coexist. In the brain, concurrent synchronization may occur at several scales, with multiple ``rhythms''…
Stochastic programming can be applied to consider uncertainties in energy system optimization models for capacity expansion planning. However, these models become increasingly large and time-consuming to solve, even without considering…
In this paper we consider distributed optimization problems in which the cost function is separable, i.e., a sum of possibly non-smooth functions all sharing a common variable, and can be split into a strongly convex term and a convex one.…
This paper investigates sensor scheduling for state estimation of complex networks over shared transmission channels. For a complex network of dynamical systems, referred to as nodes, a sensor network is adopted to measure and estimate the…
The rigid gang task model is based on the idea of executing multiple threads simultaneously on a fixed number of processors to increase efficiency and performance. Although there is extensive literature on global rigid gang scheduling,…
Partitioned Global Address Space (PGAS) integrates the concepts of shared memory programming and the control of data distribution and locality provided by message passing into a single parallel programming model. The purpose of allying…
We propose a GPU-accelerated distributed optimization algorithm for controlling multi-phase optimal power flow in active distribution systems with dynamically changing topologies. To handle varying network configurations and enable…
Many big data algorithms executed on MapReduce-like systems have a shuffle phase that often dominates the overall job execution time. Recent work has demonstrated schemes where the communication load in the shuffle phase can be traded off…
Training large machine learning (ML) models with many variables or parameters can take a long time if one employs sequential procedures even with stochastic updates. A natural solution is to turn to distributed computing on a cluster;…
We consider an ensemble of globally coupled phase oscillators whose interaction is transmitted at finite speed. This introduces time delays, which make the spatial coordinates relevant in spite of the infinite range of the interaction. We…
To train modern large DNN models, pipeline parallelism has recently emerged, which distributes the model across GPUs and enables different devices to process different microbatches in pipeline. Earlier pipeline designs allow multiple…
We describe a process calculus featuring high level constructs for component-oriented programming in a distributed setting. We propose an extension of the higher-order pi-calculus intended to capture several important mechanisms related to…
This work is a continuation of efforts to define and understand competitive analysis of algorithms in a distributed shared memory setting, which is surprisingly different from the classical online setting. In fact, in a distributed shared…
Phasor Agents are dynamical systems whose internal state is a Phasor Graph: a weighted graph of coupled Stuart-Landau oscillators. A Stuart-Landau oscillator is a minimal stable "rhythm generator" (the normal form near a Hopf bifurcation);…
In collaborative robotic cells, a human operator and a robot share the workspace in order to execute a common job, consisting of a set of tasks. A proper allocation and scheduling of the tasks for the human and for the robot is crucial for…
We present tensor and sequence parallelism (TSP), a parallel execution strategy that folds tensor parallelism and sequence parallelism onto a single device axis. In conventional multi-dimensional parallelism layouts, tensor parallelism (TP)…
In science and engineering, intelligent processing of complex signals such as images, sound or language is often performed by a parameterized hierarchy of nonlinear processing layers, sometimes biologically inspired. Hierarchical systems…
As the size of modern data sets exceeds the disk and memory capacities of a single computer, machine learning practitioners have resorted to parallel and distributed computing. Given that optimization is one of the pillars of machine…
We consider the standard broadcast setup with a single server broadcasting information to a number of clients, each of which contains local storage (called cache) of some size, which can store some parts of the available files at the…
Motivated by large-scale optimization problems arising in the context of machine learning, there have been several advances in the study of asynchronous parallel and distributed optimization methods during the past decade. Asynchronous…