English
Related papers

Related papers: On the Optimality of Coded Distributed Computing f…

200 papers

Coded computation is a framework which provides redundancy in distributed computing systems to speed up largescale tasks. Although most existing works assume an error-free scenarios in a master-worker setup, the link failures are common in…

Information Theory · Computer Science 2019-01-14 Dong-Jun Han , Jy-yong Sohn , Jaekyun Moon

Several network communication problems are highly related such as coded caching and distributed computation. The centralized coded caching focuses on reducing the network burden in peak times in a wireless network system and the coded…

Information Theory · Computer Science 2022-09-13 Min Xu , Zixiang Xu , Gennian Ge , Min-Qian Liu

We consider the problem of coded distributed computing where a large linear computational job, such as a matrix multiplication, is divided into $k$ smaller tasks, encoded using an $(n,k)$ linear code, and performed over $n$ distributed…

Information Theory · Computer Science 2019-06-25 Mohammad Vahid Jamali , Mahdi Soleymani , Hessam Mahdavifar

This paper formulates a distributed computation problem, where a master asks $N$ distributed workers to compute a linearly separable function. The task function can be expressed as $K_c$ linear combinations of $K$ messages, where each…

Information Theory · Computer Science 2021-10-26 Kai Wan , Hua Sun , Mingyue Ji , Giuseppe Caire

Motivated by mobile edge computing and wireless data centers, we study a wireless distributed computing framework where the distributed nodes exchange information over a wireless interference network. Our framework follows the structure of…

Information Theory · Computer Science 2018-10-19 Fan Li , Jinyuan Chen , Zhiying Wang

We consider the problem of coded distributed computing where a large linear computational job, such as a matrix multiplication, is divided into $k$ smaller tasks, encoded using an $(n,k)$ linear code, and performed over $n$ distributed…

Information Theory · Computer Science 2021-10-06 Mahdi Soleymani , Mohammad Vahid Jamali , Hessam Mahdavifar

We study the scalability of consensus-based distributed optimization algorithms by considering two questions: How many processors should we use for a given problem, and how often should they communicate when communication is not free?…

Distributed, Parallel, and Cluster Computing · Computer Science 2012-09-06 Konstantinos I. Tsianos , Sean Lawlor , Michael G. Rabbat

Coded Distributed Computing (CDC) introduced by Li et al. in 2015 offers an efficient approach to trade computing power to reduce the communication load in general distributed computing frameworks such as MapReduce and Spark. In particular,…

Information Theory · Computer Science 2020-07-23 Nicholas Woolsey , Rong-Rong Chen , Mingyue Ji

Gradient coding allows a master node to derive the aggregate of the partial gradients, calculated by some worker nodes over the local data sets, with minimum communication cost, and in the presence of stragglers. In this paper, for gradient…

Information Theory · Computer Science 2021-03-03 Tayyebeh Jahani-Nezhad , Mohammad Ali Maddah-Ali

Communication overhead is one of the major performance bottlenecks in large-scale distributed computing systems, in particular for machine learning applications. Conventionally, compression techniques are used to reduce the load of…

Information Theory · Computer Science 2018-05-08 Songze Li , Mohammad Ali Maddah-Ali , A. Salman Avestimehr

Coded distributed computing (CDC) introduced by Li \emph{et al.} can greatly reduce the communication load for MapReduce computing systems. In the general cascaded CDC with $K$ workers, $N$ input files and $Q$ Reduce functions, each input…

Information Theory · Computer Science 2023-11-22 Minquan Cheng , Youlong Wu , Xianxian Li , Dianhua Wu

Optimal transport has numerous applications, particularly in machine learning tasks involving generative models. In practice, the transportation process often encounters an information bottleneck, typically arising from the conversion of a…

Information Theory · Computer Science 2024-12-30 Xiqiang Qu , Ruibin Li , Jun Chen , Lei Yu , Xinbing Wang

The distributed index coding problem is studied, whereby multiple messages are stored at different servers to be broadcast to receivers with side information. First, the existing composite coding scheme is enhanced for the centralized…

Information Theory · Computer Science 2017-01-24 Yucheng Liu , Parastoo Sadeghi , Fatemeh Arbabjolfaei , Young-Han Kim

In this paper, we consider the unconstrained distributed optimization problem, in which the exchange of information in the network is captured by a directed graph topology, thus, nodes can only communicate with their neighbors.…

Systems and Control · Electrical Eng. & Systems 2023-12-07 Apostolos I. Rikos , Wei Jiang , Themistoklis Charalambous , Karl H. Johansson

We consider the "coded cooperative data exchange problem" for general graphs. In this problem, given a graph G=(V,E) representing clients in a broadcast network, each of which initially hold a (not necessarily disjoint) set of information…

Information Theory · Computer Science 2012-02-10 Mira Gonen , Michael Langberg

This thesis is concerned with the design of distributed algorithms for solving optimization problems. We consider networks where each node has exclusive access to a cost function, and design algorithms that make all nodes cooperate to find…

Optimization and Control · Mathematics 2013-12-03 João F. C. Mota

We study the joint minimization of communication and computation costs in distributed computing, where a master node coordinates $N$ workers to evaluate a function over a library of $n$ files. Assuming that the function is decomposed into…

Information Theory · Computer Science 2026-01-12 Javad Maheri , K. K. Krishnan Namboodiri , Petros Elia

We consider a standard distributed optimisation setting where $N$ machines, each holding a $d$-dimensional function $f_i$, aim to jointly minimise the sum of the functions $\sum_{i = 1}^N f_i (x)$. This problem arises naturally in…

Machine Learning · Computer Science 2021-12-08 Dan Alistarh , Janne H. Korhonen

We consider load balancing problem in a cache network consisting of storage-enabled servers forming a distributed content delivery scenario. Previously proposed load balancing solutions cannot perfectly balance out requests among servers,…

Information Theory · Computer Science 2019-08-06 Mahdi Jafari Siavoshani , Farzad Parvaresh , Ali Pourmiri , Seyed Pooya Shariatpanahi

As a strategy to further reduce the transmission pressure during the peak traffic times in wireless network, coded caching has been widely studied recently. And several coded caching schemes are constructed focusing on the two core problems…

Information Theory · Computer Science 2017-12-27 Minquan Cheng , Jing Jiang , Youzhi Yao