English
Related papers

Related papers: Repair Pipelining for Erasure-Coded Storage: Algor…

200 papers

The use of FPGAs for efficient graph processing has attracted significant interest. Recent memory subsystem upgrades including the introduction of HBM in FPGAs promise to further alleviate memory bottlenecks. However, modern multi-channel…

Hardware Architecture · Computer Science 2022-03-08 Xinyu Chen , Yao Chen , Feng Cheng , Hongshi Tan , Bingsheng He , Weng-Fai Wong

It is a challenging task to train large DNN models on sophisticated GPU platforms with diversified interconnect capabilities. Recently, pipelined training has been proposed as an effective approach for improving device utilization. However,…

Distributed, Parallel, and Cluster Computing · Computer Science 2020-07-03 Shiqing Fan , Yi Rong , Chen Meng , Zongyan Cao , Siyu Wang , Zhen Zheng , Chuan Wu , Guoping Long , Jun Yang , Lixue Xia , Lansong Diao , Xiaoyong Liu , Wei Lin

Reliability in distributed storage systems has typically focused on the design and deployment of data replication or erasure coding techniques. Although some scenarios have considered the use of replication for hot data and erasure coding…

Information Theory · Computer Science 2025-11-04 Daniel E. Lucani , Marcell Fehér

In wireless distributed storage systems, storage nodes are connected by wireless channels, which are broadcast in nature. This paper exploits this unique feature to design an efficient repair mechanism, called broadcast repair, for wireless…

Information Theory · Computer Science 2017-07-10 Ping Hu , Chi Wan Sung , Terence H. Chan

Erasure-correcting codes, that support local repair of codeword symbols, have attracted substantial attention recently for their application in distributed storage systems. This paper investigates a generalization of the usual locally…

Information Theory · Computer Science 2016-01-28 Ankit Singh Rawat , Arya Mazumdar , Sriram Vishwanath

The prioritization of restoration actions after large power system outages plays a key role in how quickly power can be restored. It has been shown that fast and intuitive heuristics for restoration prioritization most often result in…

Systems and Control · Electrical Eng. & Systems 2022-04-07 Noah Rhodes , Carleton Coffrin , Line Roald

This paper investigates the use of redundancy and self repairing against node failures in distributed storage systems, using various strategies. In replication method, access to one replication node is sufficient to reconstruct a lost node,…

Information Theory · Computer Science 2011-10-03 Abbas Kiani , Soroush Akhlaghi

A distributed storage system stores data across multiple nodes, with the primary objective of enabling efficient data recovery even in the event of node failures. The main goal of an exact repair scheme is to recover the data from a failed…

Information Theory · Computer Science 2026-01-05 Hiram H. López , Gretchen L. Matthews , Daniel Valvo

Learned image compression allows achieving state-of-the-art accuracy and compression ratios, but their relatively slow runtime performance limits their usage. While previous attempts on optimizing learned image codecs focused more on the…

Image and Video Processing · Electrical Eng. & Systems 2022-08-04 Fangzheng Lin , Heming Sun , Jiro Katto

Two widely studied models of multiple-node repair in distributed storage systems are centralized repair and cooperative repair. The centralized model assumes that all the failed nodes are recreated in one location, while the cooperative one…

Information Theory · Computer Science 2018-07-12 Min Ye , Alexander Barg

We consider distributed storage (DS) for a wireless network where mobile devices arrive and depart according to a Poisson random process. Content is stored in a number of mobile devices, using an erasure correcting code. When requesting a…

Information Theory · Computer Science 2016-01-05 Jesper Pedersen , Alexandre Graell i Amat , Iryna Andriyanova , Fredrik Brännström

Both IP lookup and packet classification in IP routers can be implemented by some form of tree traversal. SRAM-based Pipelining can improve the throughput dramatically. However, previous pipelining schemes result in unbalanced memory…

Networking and Internet Architecture · Computer Science 2011-07-28 Weirong Jiang , Hoang Le , Viktor K. Prasanna

In a distributed storage system, recovering from multiple failures is a critical and frequent task that is crucial for maintaining the system's reliability and fault-tolerance. In this work, we focus on the problem of repairing multiple…

Information Theory · Computer Science 2018-05-09 Marwen Zorgui , Zhiying Wang

Reed-Solomon codes have found many applications in practical storage systems, but were until recently considered unsuitable for distributed storage applications due to the widely-held belief that they have poor repair bandwidth. The work of…

Information Theory · Computer Science 2017-05-16 Hoang Dau , Olgica Milenkovic

Regenerating codes are efficient methods for distributed storage in storage networks, where node failures are common. They guarantee low cost data reconstruction and repair through accessing only a predefined number of arbitrarily chosen…

Information Theory · Computer Science 2017-11-09 Kaveh Mahdaviani , Ashish Khisti , Soheil Mohajer

Network codes designed specifically for distributed storage systems have the potential to provide dramatically higher storage efficiency for the same availability. One main challenge in the design of such codes is the exact repair problem:…

Information Theory · Computer Science 2011-09-02 Dimitris S. Papailiopoulos , Jianqiang Luo , Alexandros G. Dimakis , Cheng Huang , Jin Li

Erasure coding has been recognized as a powerful method to mitigate delays due to slow or straggling nodes in distributed systems. This work shows that erasure coding of data objects can flexibly handle skews in the request rates. Coding…

Information Theory · Computer Science 2021-06-29 Mehmet Aktas , Gauri Joshi , Swanand Kadhe , Fatemeh Kazemi , Emina Soljanin

Deep neural networks with large model sizes achieve state-of-the-art results for tasks in computer vision (CV) and natural language processing (NLP). However, these large-scale models are too compute- or memory-intensive for…

Distributed, Parallel, and Cluster Computing · Computer Science 2021-10-29 Yang Hu , Connor Imes , Xuanang Zhao , Souvik Kundu , Peter A. Beerel , Stephen P. Crago , John Paul N. Walters

The piggybacking framework for designing erasure codes for distributed storage has empirically proven to be very useful, and has been used to design codes with desirable properties, such as low repair bandwidth and complexity. However, the…

Information Theory · Computer Science 2017-07-11 Reyna Hulett , Mary Wootters

With the increasing scale of models, the need for efficient distributed training has become increasingly urgent. Recently, many synchronous pipeline parallelism approaches have been proposed to improve training throughput. However, these…

Machine Learning · Computer Science 2024-10-28 Houming Wu , Ling Chen , Wenjie Yu