Related papers: Distributed Perceptron under Bounded Staleness, Pa…
With the increasing demand for large-scale training of machine learning models, consensus-based distributed optimization methods have recently been advocated as alternatives to the popular parameter server framework. In this paradigm, each…
Federated learning is a communication-efficient training process that alternates between local training at the edge devices and averaging the updated local model at the central server. Nevertheless, it is impractical to achieve a perfect…
Regional energy caps limit the growth of any single data center used for large-scale model training. This single-center training paradigm works when model size remains manageable, but exponential growth in the model size and computational…
Federated learning (FL) is a distributed machine learning paradigm that enables multiple clients to train a shared model collaboratively while preserving privacy. However, the scaling of real-world FL systems is often limited by two…
We study distributed estimation of a high-dimensional static parameter vector through a group of sensors whose communication network is modeled by a fixed directed graph. Different from existing time-triggered communication schemes, an…
In this work we study the problem of unconstrained convex-optimization in a fully distributed multi-agent setting which includes asynchronous computation and lossy communication. In particular, we extend a recently proposed algorithm named…
Distributed computing systems often consist of hundreds of nodes, executing tasks with different resource requirements. Efficient resource provisioning and task scheduling in such systems are non-trivial and require close monitoring and…
We study infinite-horizon asymptotic average optimality for parallel server network with multiple classes of jobs and multiple server pools in the Halfin-Whitt regime. Three control formulations are considered: 1) minimizing the queueing…
Federated learning enables collaborative model training across distributed clients while preserving data privacy. However, in practical deployments, device heterogeneity, non-independent, and identically distributed (Non-IID) data often…
Online decision systems routinely operate under delayed feedback and order-sensitive (noncommutative) dynamics: actions affect which observations arrive, and in what sequence. Taking a Bregman divergence $D_\Phi$ as the loss benchmark, we…
This paper studies how communication across experts prior to aggregation by a decision-maker affects the efficiency of forecast combination. When experts exchange information before reporting their forecasts, their signals become correlated…
In Part II [3] we carried out a detailed mean-square-error analysis of the performance of asynchronous adaptation and learning over networks under a fairly general model for asynchronous events including random topologies, random link…
We investigate the effect of noise on Random Boolean Networks. Noise is implemented as a probability $p$ that a node does not obey its deterministic update rule. We define two order parameters, the long-time average of the Hamming distance…
We study online federated learning over a wireless network, where the central server updates an online global model sequence to minimize the time-varying loss of multiple local devices over time. The server updates the global model through…
Communication is a crucial phase in the context of distributed training. Because parameter server (PS) frequently experiences network congestion, recent studies have found that training paradigms without a centralized server outperform the…
We study distributed stochastic convex optimization under the delayed gradient model where the server nodes perform parameter updates, while the worker nodes compute stochastic gradients. We discuss, analyze, and experiment with a setup…
Wireless sensing and the internet of things (IoT) are nowadays pervasive in 5G and beyond networks, and they are expected to play a crucial role in 6G. However, a centralized optimization of a distributed system is not always possible and…
In a state-update protocol for a system of $L$ asynchronous parallel processes that communicate only with nearest neighbors, global desynchronization in operation times can be deduced from kinetic roughening of the corresponding…
Many machine learning algorithms have been developed under the assumption that data sets are already available in batch form. Yet in many application domains data is only available sequentially overtime via compute nodes in different…
Distributed optimization finds applications in large-scale machine learning, data processing and classification over multi-agent networks. In real-world scenarios, the communication network of agents may encounter latency that may affect…