English
Related papers

Related papers: Deterministic Data Distribution for Efficient Reco…

200 papers

We consider distributed storage (DS) for a wireless network where mobile devices arrive and depart according to a Poisson random process. Content is stored in a number of mobile devices, using an erasure correcting code. When requesting a…

Information Theory · Computer Science 2016-01-05 Jesper Pedersen , Alexandre Graell i Amat , Iryna Andriyanova , Fredrik Brännström

Scientific computing workflows generate enormous distributed data that is short-lived, yet critical for job completion time. This class of data is called intermediate data. A common way to achieve high data availability is to replicate…

Distributed, Parallel, and Cluster Computing · Computer Science 2020-04-14 Zhe Zhang , Brian Bockelman , Derek Weitzel , David Swanson

We examine the problem of creating an encoded distributed storage representation of a data object for a network of mobile storage nodes so as to achieve the optimal recovery delay. A source node creates a single data object and disseminates…

Information Theory · Computer Science 2016-11-17 Derek Leong , Alexandros G. Dimakis , Tracey Ho

In this paper, we study distributed storage problems over unidirectional ring networks, whose storage nodes form a directed ring and data is transmitted along the same direction. The original data is distributed to store on these nodes.…

Information Theory · Computer Science 2014-04-15 Jiyong Lu , Xuan Guang , Fang-Wei Fu

In a distributed storage systems (DSS), regenerating codes are used to optimize bandwidth in the repair process of a failed node. To optimize other DSS parameters such as computation and disk I/O, Distributed Replication-based Simple…

Information Theory · Computer Science 2013-03-28 Srijan Anil , Manish K. Gupta , T. Aaron Gulliver

With the increase of huge amounts of data in volume, velocity, and variety, the need for capacity of Redundant Arrays of Inexpensive Disks (RAID) storage systems is dramatically growing. However, the probability of disk failures in RAID…

Distributed, Parallel, and Cluster Computing · Computer Science 2017-09-15 De-zhai Yuan , Xing-yi Peng , Ting Liu , Zhe Cui

Modern distributed storage systems offer large capacity to satisfy the exponentially increasing need of storage space. They often use erasure codes to protect against disk and node failures to increase reliability, while trying to meet the…

Distributed, Parallel, and Cluster Computing · Computer Science 2017-08-04 Yu Xiang , Tian Lan , Vaneet Aggarwal , Yih-Farn R Chen

One of the primary objectives of a distributed storage system is to reliably store large amounts of source data for long durations using a large number $N$ of unreliable storage nodes, each with $c$ bits of storage capacity. Storage nodes…

Information Theory · Computer Science 2021-01-14 Michael Luby , Thomas Richardson

Distributed storage systems for large clusters typically use replication to provide reliability. Recently, erasure codes have been used to reduce the large storage overhead of three-replicated systems. Reed-Solomon codes are the standard…

Classical erasure codes, e.g. Reed-Solomon codes, have been acknowledged as an efficient alternative to plain replication to reduce the storage overhead in reliable distributed storage systems. Yet, such codes experience high overhead…

Distributed, Parallel, and Cluster Computing · Computer Science 2012-06-20 Anne-Marie Kermarrec , Erwan Le Merrer , Gilles Straub , Alexandre van Kempen

We introduce a new class of exact Minimum-Bandwidth Regenerating (MBR) codes for distributed storage systems, characterized by a low-complexity uncoded repair process that can tolerate multiple node failures. These codes consist of the…

Information Theory · Computer Science 2010-10-14 Salim El Rouayheb , Kannan Ramchandran

We explore the feasibility of implementing a reliable, high performance, distributed storage system on a commodity computing cluster. Files are distributed across storage nodes using erasure coding with small Low-Density Parity-Check (LDPC)…

Distributed, Parallel, and Cluster Computing · Computer Science 2007-05-23 Benjamin Gaidioz , Birger Koblitz , Nuno Santos

We study the repair problem of distributed storage systems in erasure networks where the packets transmitted from surviving nodes to the new node might be lost. The fundamental storage-bandwidth tradeoff is calculated by multicasting…

Information Theory · Computer Science 2013-01-30 Majid Gerami , Ming Xiao

We introduce the Fixed Cluster Repair System (FCRS) as a novel architecture for Distributed Storage Systems (DSS), achieving a small repair bandwidth while guaranteeing a high availability. Specifically we partition the set of servers in a…

Information Theory · Computer Science 2019-03-06 Saeid Sahraei , Michael Gastpar

To achieve reliability in distributed storage systems, data has usually been replicated across different nodes. However the increasing volume of data to be stored has motivated the introduction of erasure codes, a storage efficient…

Distributed, Parallel, and Cluster Computing · Computer Science 2012-08-06 Lluis Pamies-Juarez , Anwitaman Datta , Frederique Oggier

In a distributed storage systems (DSS) with $k$ systematic nodes, robustness against node failure is commonly provided by storing redundancy in a number of other nodes and performing repair mechanism to reproduce the content of the failed…

Information Theory · Computer Science 2018-01-01 Kaveh Mahdaviani , Soheil Mohajer , Ashish Khisti

In large-scale distributed storage systems (DSS), reliability is provided by redundancy spread over storage servers across the Internet. Network coding (NC) has been widely studied in DSS because it can improve the reliability with low…

Distributed, Parallel, and Cluster Computing · Computer Science 2015-11-17 Chengjin Jia , Jin Wang , Yanqin Zhu , Xin Wang , Kejie Lu , Xiumin Wang , Zhengqing Wen

We study the problem of storing a data object in a set of data nodes that fail independently with given probabilities. Our problem is a natural generalization of a homogenous storage allocation problem where all the nodes had the same…

Information Theory · Computer Science 2012-02-09 Vasileios Ntranos , Giuseppe Caire , Alexandros G. Dimakis

Heterogeneous Distributed Storage Systems (DSSs) are close to the real world applications for data storage. Each node of the considered DSS, may store different number of packets and each having different repair bandwidth with uniform…

Information Theory · Computer Science 2016-07-22 Krishna Gopal Benerjee , Manish K. Gupta

The explosion in the volumes of data being stored online has resulted in distributed storage systems transitioning to erasure coding based schemes. Yet, the codes being deployed in practice are fairly short. In this work, we address what we…

Information Theory · Computer Science 2016-09-22 Parikshit Gopalan , Guangda Hu , Swastik Kopparty , Shubhangi Saraf , Carol Wang , Sergey Yekhanin