Related papers: Bandwidth Cost of Code Conversions in Distributed …
Distributed storage systems must store large amounts of data over long periods of time. To avoid data loss due to device failures, an $[n,k]$ erasure code is used to encode $k$ data symbols into a codeword of $n$ symbols that are stored…
Recent advances in erasure coding for distributed storage systems have demonstrated that adapting redundancy to varying disk failure rates can lead to substantial storage savings. Such adaptation requires code conversion, wherein data…
Large-scale distributed storage systems typically use erasure codes to provide durability of data in the face of failures. A set of $k$ blocks to be stored is encoded using an $[n, k]$ code to generate $n$ blocks that are then stored on…
In large-scale distributed storage systems, erasure coding is employed to ensure reliability against disk failures. Recent work by Kadekodi et al. demonstrates that adapting code parameters to varying disk failure rates can lead to…
In large-scale distributed storage systems, erasure codes are used to achieve fault tolerance in the face of node failures. Tuning code parameters to observed failure rates has been shown to significantly reduce storage cost. Such tuning of…
Recent studies have shown that distributed storage systems can achieve significant space savings by adapting redundancy levels to varying disk failure rates. This adaptation is performed via code conversion, wherein data encoded under an…
Most large-scale storage systems employ erasure coding to provide resilience against disk failures. Recent work has shown that tuning this redundancy to changes in disk failure rates leads to substantial storage savings. This process…
Distributed storage systems often introduce redundancy to increase reliability. When coding is used, the repair problem arises: if a node storing encoded information fails, in order to maintain the same level of reliability we need to…
In a distributed storage system based on erasure coding, an important problem is the \emph{repair problem}: If a node storing a coded piece fails, in order to maintain the same level of reliability, we need to create a new encoded piece and…
Modern large-scale distributed storage systems use erasure codes to protect against node failures with low storage overhead. In practice, the failure rate and other factors of storage devices in the system may vary significantly over time,…
Distributed storage systems provide reliable access to data through redundancy spread over individually unreliable nodes. Application scenarios include data centers, peer-to-peer storage systems, and storage in wireless networks. Storing…
Maximum distance separable (MDS) codes facilitate the achievement of elevated levels of fault tolerance in storage systems while incurring minimal redundancy overhead. Reed-Solomon (RS) codes are typical MDS codes with the sub-packetization…
We propose several new lower bounds on the bandwidth costs of MDS convertible codes using a linear-algebraic framework. The derived bounds improve previous results in certain parameter regimes and match the bandwidth cost of the…
Regenerating codes are efficient methods for distributed storage in storage networks, where node failures are common. They guarantee low cost data reconstruction and repair through accessing only a predefined number of arbitrarily chosen…
Error-correcting codes are essential for ensuring fault tolerance in modern distributed data storage systems. However, in practice, factors such as the failure rates of storage devices can vary significantly over time, resulting in changes…
Erasure coding techniques are used to increase the reliability of distributed storage systems while minimizing storage overhead. Also of interest is minimization of the bandwidth required to repair the system following a node failure. In a…
Distributed storage systems must handle both data heterogeneity, arising from non-uniform access demands, and device heterogeneity, caused by time-varying node reliability. In this paper, we study convertible codes, which enable the…
We propose a generic transformation that can convert any nonbinary $(n=k+r,k)$ maximum distance separable (MDS) code into another $(n,k)$ MDS code over the same field such that 1) some arbitrarily chosen $r$ nodes have the optimal repair…
Training a machine learning model is both compute and data-intensive. Most of the model training is performed on high performance compute nodes and the training data is stored near these nodes for faster training. But there is a growing…
Regenerating codes for distributed storage have attracted much research interest in the past decade. Such codes trade the bandwidth needed to repair a failed node with the overall amount of data stored in the network. Minimum storage…