Related papers: Short-circuiting Rings for Low-Latency AllReduce
Messages in communication networks often are considered as "discrete" taking values in some finite alphabet (e.g. a finite field). However, if we want to consider for example communication based on analogue signals, we will have to consider…
Hybrid Automatic Repeat Request (HARQ) schemes typically allocate all available resources to retransmit failed packets to ensure reliability. However, under stringent delay constraints, these schemes often exhibit low spectral efficiency…
In a prior work (ICDCS 2017), we presented a distributed self-adjusting algorithm DSG for skip graphs. DSG performs topological adaption to communication pattern to minimize the average routing costs between communicating nodes. In this…
Recursive least squares (RLS) algorithms were once widely used for training small-scale neural networks, due to their fast convergence. However, previous RLS algorithms are unsuitable for training deep neural networks (DNNs), since they…
Isomorphic (sparse) collective communication is a form of collective communication in which all involved processes communicate in small, identically structured neighborhoods of other processes. Isomorphic neighborhoods are defined via an…
In this paper, we investigate a novel minimum length scheduling problem to determine the optimal power control, and scheduling for constant and continuous rate models, while considering concurrent transmission of users, energy causality,…
We develop new routing algorithms for a quantum network with noisy quantum devices such that each can store a small number of qubits. We thereby consider two models for the operation of such a network. The first is a continuous model, in…
Distributed training of foundation models via $\texttt{DDP}$ is limited by interconnect bandwidth. While infrequent communication strategies reduce synchronization frequency, they remain bottlenecked by the memory and communication…
Consider a distributed computing system in which the worker nodes are connected over a shared wireless channel. Nodes can store a fraction of the data set over which computation needs to be carried out, and a Map-Shuffle-Reduce protocol is…
We consider the problem where $M$ agents interact with $M$ identical and independent environments with $S$ states and $A$ actions using reinforcement learning for $T$ rounds. The agents share their data with a central server to minimize…
A scalable algorithm for solving compact banded linear systems on distributed memory architectures is presented. The proposed method factorizes the original system into two levels of memory hierarchies, and solves it using parallel cyclic…
Hierarchical ring networks, which hierarchically connect multiple levels of rings, have been proposed in the past to improve the scalability of ring interconnects, but past hierarchical ring designs sacrifice some of the key benefits of…
We investigate optimal routing and scheduling strategies for multi-hop wireless networks with rateless codes. Rateless codes allow each node of the network to accumulate mutual information from every packet transmission. This enables a…
Assuming noisy feedback channels, this paper investigates the data transmission efficiency and robustness of different automatic repeat request (ARQ) schemes using adaptive power allocation. Considering different block-fading channel…
We consider the problem of how to reduce the cost of communication that is required for the parallel training of a neural network. The state-of-the-art method, Bulk Synchronous Parallel Stochastic Gradient Descent (BSP-SGD), requires many…
The efficient implementation of collective communiction operations has received much attention. Initial efforts produced "optimal" trees based on network communication models that assumed equal point-to-point latencies between any two…
We show communication schedulers' recent work proposed for ML collectives does not scale to the increasing problem sizes that arise from training larger models. These works also often produce suboptimal schedules. We make a connection with…
Future 5G cellular networks supporting ultra-reliable, low-latency communications (URLLC) could employ random access communication to reduce the overhead compared to scheduled access techniques used in 4G networks. We consider a wireless…
Oblivious routing has a long history in both the theory and practice of networking. In this work we initiate the formal study of oblivious routing in the context of reconfigurable networks, a new architecture that has recently come to the…
This work presents joint interference suppression and power allocation algorithms for DS-CDMA networks with multiple hops and decode-and-forward (DF) protocols. A scheme for joint allocation of power levels across the relays subject to…