English
Related papers

Related papers: Serverless Straggler Mitigation using Local Error-…

200 papers

Coded polynomial aggregation (CPA) in distributed computing systems enables the master to directly recover a weighted aggregation of polynomial computations without individually decoding each term, thereby reducing the number of required…

Information Theory · Computer Science 2026-02-04 Xi Zhong , Jörg Kliewer , Mingyue Ji

Modern distributed storage systems offer large capacity to satisfy the exponentially increasing need of storage space. They often use erasure codes to protect against disk and node failures to increase reliability, while trying to meet the…

Distributed, Parallel, and Cluster Computing · Computer Science 2017-08-04 Yu Xiang , Tian Lan , Vaneet Aggarwal , Yih-Farn R Chen

We consider the problem of tracking the state of a process that evolves over time in a distributed setting, with multiple observers each observing parts of the state, which is a fundamental information processing problem with a wide range…

Information Theory · Computer Science 2019-09-04 Albin Severinson , Eirik Rosnes , Alexandre Graell i Amat

To improve the utility of learning applications and render machine learning solutions feasible for complex applications, a substantial amount of heavy computations is needed. Thus, it is essential to delegate the computations among several…

Distributed, Parallel, and Cluster Computing · Computer Science 2022-04-29 Homa Esfahanizadeh , Alejandro Cohen , Muriel Medard

Coded matrix multiplication is a technique to enable straggler-resistant multiplication of large matrices in distributed computing systems. In this paper, we first present a conceptual framework to represent the division of work amongst…

Information Theory · Computer Science 2019-07-23 Shahrzad Kiani , Nuwan Ferdinand , Stark C. Draper

Placement delivery arrays for distributed computing (Comp-PDAs) have recently been proposed as a framework to construct universal computing schemes for MapReduce-like systems. In this work, we extend this concept to systems with straggling…

Information Theory · Computer Science 2020-04-28 Qifa Yan , Michèle Wigger , Sheng Yang , Xiaohu Tang

In this paper, we consider the iterative solution of linear algebraic equations under the condition that matrix-vector products with the coefficient matrix are computed only partially. At the same time, non-computed entries are set to…

Numerical Analysis · Mathematics 2024-10-15 Vassilis Kalantzis , Yuanzhe Xi , Lior Horesh , Yousef Saad

One of the major challenges in using distributed learning to train complicated models with large data sets is to deal with stragglers effect. As a solution, coded computation has been recently proposed to efficiently add redundancy to the…

Information Theory · Computer Science 2021-11-02 Tayyebeh Jahani-Nezhad , Mohammad Ali Maddah-Ali

We consider the problem of secure distributed matrix computation (SDMC), where a \textit{user} queries a function of data matrices generated at distributed \textit{source} nodes. We assume the availability of $N$ honest but curious…

Information Theory · Computer Science 2021-11-16 Nitish Mital , Cong Ling , Deniz Gunduz

The distributed linearly separable computation problem finds extensive applications across domains such as distributed gradient coding, distributed linear transform, real-time rendering, etc. In this paper, we investigate this problem in a…

Information Theory · Computer Science 2024-01-30 Haoning Chen , Minquan Cheng , Zhenhao Huang , Youlong Wu

Today's data centers have an abundance of computing resources, hosting server clusters consisting of as many as tens or hundreds of thousands of machines. To execute a complex computing task over a data center, it is natural to distribute…

Information Theory · Computer Science 2017-02-24 Qian Yu , Songze Li , Mohammad Ali Maddah-Ali , A. Salman Avestimehr

To enhance straggler resilience in federated learning (FL) systems, a semi-decentralized approach has been recently proposed, enabling collaboration between clients. Unlike the existing semi-decentralized schemes, which adaptively adjust…

Information Theory · Computer Science 2025-02-20 Shudi Weng , Ming Xiao , Chao Ren , Mikael Skoglund

We consider straggler-resilient learning. In many previous works, e.g., in the coded computing literature, straggling is modeled as random delays that are independent and identically distributed between workers. However, in many practical…

Distributed, Parallel, and Cluster Computing · Computer Science 2021-11-30 Albin Severinson , Eirik Rosnes , Salim El Rouayheb , Alexandre Graell i Amat

Cloud computing facilitates the access of applications and data from any location by a distributed storage system. Erasure codes offer better data replication technique with reduced storage costs for more reliability. This paper considers…

Information Theory · Computer Science 2021-02-04 Srujan Teja Thomdapu , Ketan Rajawat

We propose a novel application of coded computing to the problem of the nearest neighbor estimation using MatDot Codes [Fahim. et.al. 2017], that are known to be optimal for matrix multiplication in terms of recovery threshold under storage…

Information Theory · Computer Science 2018-11-30 Utsav Sheth , Sanghamitra Dutta , Malhar Chaudhari , Haewon Jeong , Yaoqing Yang , Jukka Kohonen , Teemu Roos , Pulkit Grover

We propose a novel coding theoretic framework for mitigating stragglers in distributed learning. We show how carefully replicating data blocks and coding across gradients can provide tolerance to failures and stragglers for Synchronous…

Machine Learning · Statistics 2017-03-09 Rashish Tandon , Qi Lei , Alexandros G. Dimakis , Nikos Karampatziakis

We present a novel coded federated learning (FL) scheme for linear regression that mitigates the effect of straggling devices while retaining the privacy level of conventional FL. The proposed scheme combines one-time padding to preserve…

Machine Learning · Computer Science 2022-02-16 Siddhartha Kumar , Reent Schlegel , Eirik Rosnes , Alexandre Graell i Amat

We consider the distributed computing framework of MapReduce, which consists of three phases, the Map phase, the Shuffle phase and the Reduce phase. For this framework, we propose the use of binary matrices (with $0,1$ entries) called…

Information Theory · Computer Science 2020-02-03 Shailja Agrawal , Prasad Krishnan

Distributed stochastic gradient descent (SGD) is essential for scaling the machine learning algorithms to a large number of computing nodes. However, the infrastructures variability such as high communication delay or random node slowdown…

Machine Learning · Computer Science 2020-02-25 Jianyu Wang , Hao Liang , Gauri Joshi

Raptor codes are rateless codes that achieve the capacity on the binary erasure channels. However the maximum degree of optimal output degree distribution is unbounded. This leads to a computational complexity problem both at encoders and…

Information Theory · Computer Science 2013-02-07 Kosuke Sakata , Kenta Kasai , Kohichi Sakaniwa