English
Related papers

Related papers: Deterministic Data Distribution for Efficient Reco…

200 papers

In large data centers, storage nodes are organized in racks, and the cross-rack transmission dominates the bandwidth cost. For the repair of single node failures, codes achieving the tradeoff between the storage redundancy and cross-rack…

Information Theory · Computer Science 2022-05-19 Liyang Zhou , Zhifang Zhang

In distributed storage systems that employ erasure coding, the issue of minimizing the total {\it communication} required to exactly rebuild a storage node after a failure arises. This repair bandwidth depends on the structure of the…

Information Theory · Computer Science 2011-06-09 Dimitris S. Papailiopoulos , Alexandros G. Dimakis , Viveck R. Cadambe

Distributed RDF systems partition data across multiple computer nodes (workers). Some systems perform cheap hash partitioning, which may result in expensive query evaluation, while others apply heuristics aiming at minimizing inter-node…

Databases · Computer Science 2015-05-12 Razen Harbi , Ibrahim Abdelaziz , Panos Kalnis , Nikos Mamoulis , Yasser Ebrahim , Majed Sahli

In cloud storage systems with a large number of servers, files are typically not stored in single servers. Instead, they are split, replicated (to ensure reliability in case of server malfunction) and stored in different servers. We analyze…

Distributed, Parallel, and Cluster Computing · Computer Science 2018-07-09 Avishek Ghosh , Kannan Ramchandran

Storage systems have a strong need for substantially improving their error correction capabilities, especially for long-term storage where the accumulating errors can exceed the decoding threshold of error-correcting codes (ECCs). In this…

Information Theory · Computer Science 2018-11-12 Pulakesh Upadhyaya , Anxiao , Jiang

Distributed systems store data objects redundantly to balance the data access load over multiple nodes. Load balancing performance depends mainly on 1) the level of storage redundancy and 2) the assignment of data objects to storage nodes.…

Distributed, Parallel, and Cluster Computing · Computer Science 2023-12-19 Mehmet Aktas , Emina Soljanin

HPC systems are a critical resource for scientific research. The increased demand for computational power and memory ushers in the exascale era, in which supercomputers are designed to provide enormous computing power to meet these needs.…

Distributed, Parallel, and Cluster Computing · Computer Science 2022-08-11 Yehonatan Fridman , Yaniv Snir , Harel Levin , Danny Hendler , Hagit Attiya , Gal Oren

When deploying fast charging stations (FCSs) to support long-distance trips of electric vehicles (EVs), there exist indirect network effects: while the gradual diffusion of EVs directly influences the timing and capacities of FCS…

Systems and Control · Electrical Eng. & Systems 2023-10-13 Yujia Li , Feng Qiu , Yixuan Chen , Yunhe Hou

Network transfer and disk read are the most time consuming operations in the repair process for node failures in erasure-code-based distributed storage systems. Recent developments on Reed-Solomon codes, the most widely used erasure codes…

Information Theory · Computer Science 2018-05-10 Hoang Dau , Iwan Duursma , Hien Chu

We consider the setting of data storage across n nodes in a distributed manner. A data collector (DC) should be able to reconstruct the entire data by connecting to any k out of the n nodes and downloading all the data stored in them. When…

Information Theory · Computer Science 2009-09-05 Nihar B. Shah , K. V. Rashmi , P. Vijay Kumar , Kannan Ramchandran

We consider the use of distributed storage (DS) to reduce the communication cost of content delivery in wireless networks. Content is stored (cached) in a number of mobile devices using an erasure correcting code. Users retrieve content…

Information Theory · Computer Science 2016-09-28 Jesper Pedersen , Alexandre Graell i Amat , Iryna Andriyanova , Fredrik Brännström

The effect of redundancy on the aging of an efficient Maximum Distance Separable (MDS) parity--protected distributed storage system that consists of multidimensional arrays of storage units is explored. In light of the experimental…

Information Theory · Computer Science 2013-12-04 Suayb S. Arslan

The problem of securing data against eavesdropping in distributed storage systems is studied. The focus is on systems that use linear codes and implement exact repair to recover from node failures.The maximum file size that can be stored…

Information Theory · Computer Science 2013-04-15 Sreechakra Goparaju , Salim El Rouayheb , Robert Calderbank , H. Vincent Poor

We present a distributionally robust optimization (DRO) approach for the transmission expansion planning problem, considering both long- and short-term uncertainties on the system demand and non-dispatchable renewable generation. On the…

Optimization and Control · Mathematics 2020-03-17 Alexandre Velloso , David Pozo , Alexandre Street

As rapidly growing AI computational demands accelerate the need for new hardware installation and maintenance, this work explores optimal data center resource management by balancing operational efficiency with fault tolerance through…

Artificial Intelligence · Computer Science 2025-04-02 Chang-Lin Chen , Jiayu Chen , Tian Lan , Zhaoxia Zhao , Hongbo Dong , Vaneet Aggarwal

Minimum storage regenerating (MSR) codes, with the MDS property and the optimal repair bandwidth, are widely used in distributed storage systems (DSS) for data recovery. In this paper, we consider the construction of $(n,k,l)$ MSR codes in…

Information Theory · Computer Science 2023-09-28 Shenghua Li , Maximilien Gadouleau , Jiaojiao Wang , Dabin Zheng

Regenerating codes are a class of recently developed codes for distributed storage that, like Reed-Solomon codes, permit data recovery from any subset of k nodes within the n-node network. However, regenerating codes possess in addition,…

Information Theory · Computer Science 2015-03-17 Nihar B. Shah , K. V. Rashmi , P. Vijay Kumar , Kannan Ramchandran

In this paper, given a random uniform distribution of sensor nodes on a 2-D plane, a fast self-organized distributed algorithm is proposed to find the maximum number of partitions of the nodes such that each partition is connected and…

Distributed, Parallel, and Cluster Computing · Computer Science 2013-06-12 Dibakar Saha , Nabanita Das

We propose a framework for distributed robust statistical learning on {\em big contaminated data}. The Distributed Robust Learning (DRL) framework can reduce the computational time of traditional robust learning methods by several orders of…

Machine Learning · Statistics 2015-02-10 Jiashi Feng , Huan Xu , Shie Mannor

Erasure coding has been recognized as a powerful method to mitigate delays due to slow or straggling nodes in distributed systems. This work shows that erasure coding of data objects can flexibly handle skews in the request rates. Coding…

Information Theory · Computer Science 2021-06-29 Mehmet Aktas , Gauri Joshi , Swanand Kadhe , Fatemeh Kazemi , Emina Soljanin