English
Related papers

Related papers: XORing Elephants: Novel Erasure Codes for Big Data

200 papers

The problem of repairing linear codes and, in particular, Reed Solomon (RS) codes has attracted a lot of attention in recent years due to their extreme importance to distributed storage systems. In this problem, a failed code symbol (node)…

Information Theory · Computer Science 2022-04-12 Roni Con , Itzhak Tamo

Reed-Solomon (RS) codes are widely used in distributed storage systems. In this paper, we study the repair bandwidth and sub-packetization size of RS codes. The repair bandwidth is defined as the amount of transmitted information from…

Information Theory · Computer Science 2019-03-11 Weiqi Li , Zhiying Wang , Hamid Jafarkhani

Maximum distance separable (MDS) codes facilitate the achievement of elevated levels of fault tolerance in storage systems while incurring minimal redundancy overhead. Reed-Solomon (RS) codes are typical MDS codes with the sub-packetization…

Information Theory · Computer Science 2024-05-07 Hao Shi , Zhengyi Jiang , Zhongyi Huang , Bo Bai , Gong Zhang , Hanxu Hou

A distributed storage system stores data across multiple nodes, with the primary objective of enabling efficient data recovery even in the event of node failures. The main goal of an exact repair scheme is to recover the data from a failed…

Information Theory · Computer Science 2026-01-05 Hiram H. López , Gretchen L. Matthews , Daniel Valvo

This work presents a theoretical analysis of the probability of successfully retrieving data encoded with MDS codes (e.g., Reed-Solomon codes) in DNA storage systems. We study this probability under independent and identically distributed…

Information Theory · Computer Science 2026-05-08 Serge Kas Hanna

Considerable interest has been paid in recent literature to codes combining local and global properties for erasure correction. Applications are in cloud type of implementations, in which fast recovery of a failed storage device is…

Information Theory · Computer Science 2016-02-09 Mario Blaum , Steven R. Hetzler

A scheme for concatenating the recently invented polar codes with non-binary MDS codes, as Reed-Solomon codes, is considered. By concatenating binary polar codes with interleaved Reed-Solomon codes, we prove that the proposed concatenation…

Information Theory · Computer Science 2016-11-17 Hessam Mahdavifar , Mostafa El-Khamy , Jungwon Lee , Inyup Kang

Modern distributed storage systems apply redundancy coding techniques to stored data. One form of redundancy is based on regenerating codes, which can minimize the repair bandwidth, i.e., the amount of data transferred when repairing a…

Information Theory · Computer Science 2013-01-23 Yuchong Hu , Patrick P. C. Lee , Kenneth W. Shum

In this paper, we propose locally repairable codes (LRCs) with optimal minimum distance for distributed storage systems (DSS). A two-layer encoding structure is employed to ensure data reconstruction and the designated repair locality. The…

Information Theory · Computer Science 2014-08-15 Hongmei Xie , Zhiyuan Yan

Motivated by emerging applications to the edge computing paradigm, we introduce a two-layer erasure-coded fault-tolerant distributed storage system offering atomic access for read and write operations. In edge computing, clients interact…

Distributed, Parallel, and Cluster Computing · Computer Science 2017-06-01 Kishori M. Konwar , N. Prakash , Nancy Lynch , Muriel Medard

Multishot network coding is considered in a worst-case adversarial setting in which an omniscient adversary with unbounded computational resources may inject erroneous packets in up to $t$ links, erase up to $\rho$ packets, and wire-tap up…

Information Theory · Computer Science 2019-04-18 Umberto Martínez-Peñas , Frank R. Kschischang

Locally repairable codes (LRCs) are considered with equal or unequal localities, local distances and local field sizes. An explicit two-layer architecture with a sum-rank outer code is obtained, having disjoint local groups and achieving…

Information Theory · Computer Science 2019-04-25 Umberto Martínez-Peñas , Frank R. Kschischang

This paper presents two repair schemes for low-rate Reed-Solomon (RS) codes over prime fields that can repair any node by downloading a constant number of bits from each surviving node. The total bandwidth resulting from these schemes is…

Information Theory · Computer Science 2024-02-06 Roni Con , Noah Shutty , Itzhak Tamo , Mary Wootters

We study the repair problem of distributed storage systems in erasure networks where the packets transmitted from surviving nodes to the new node might be lost. The fundamental storage-bandwidth tradeoff is calculated by multicasting…

Information Theory · Computer Science 2013-01-30 Majid Gerami , Ming Xiao

Erasure coding (EC) affords data redundancy for large-scale systems. XOR-based EC is an easy-to-implement method for optimizing EC. This paper addresses a significant performance gap between the state-of-the-art XOR-based EC approach (with…

Programming Languages · Computer Science 2021-08-06 Yuya Uezato

In this paper, we address the node repair problem of Reed-Solomon (RS) coded distributed storage systems. Specifically, to overcome the challenges of multiple-node failures of RS codes under the rack-aware storage model, we employ good…

Information Theory · Computer Science 2024-05-28 Yumeng Yang , Han Cai , Xiaohu Tang

In a modern distributed storage system, storage nodes are organized in racks, and the cross-rack communication dominates the system bandwidth. In We study the rack-aware storage system where all storage nodes are organized in racks and…

Information Theory · Computer Science 2022-07-18 Liyang Zhou , Zhifang Zhang

In this paper, we prove that with high probability, random Reed-Solomon codes approach the half-Singleton bound - the optimal rate versus error tradeoff for linear insdel codes - with linear-sized alphabets. More precisely, we prove that,…

Information Theory · Computer Science 2024-07-11 Roni Con , Zeyu Guo , Ray Li , Zihan Zhang

The problem of securing data against eavesdropping in distributed storage systems is studied. The focus is on systems that use linear codes and implement exact repair to recover from node failures.The maximum file size that can be stored…

Information Theory · Computer Science 2013-04-15 Sreechakra Goparaju , Salim El Rouayheb , Robert Calderbank , H. Vincent Poor

There are different ways to realize Reed Solomon (RS) codes. While in the storage community, using the generator matrices to implement RS codes is more popular, in the coding theory community the generator polynomials are typically used to…

Information Theory · Computer Science 2013-12-19 Kyumars Sheykh Esmaili , Anwitaman Datta