Related papers: Storage-Optimized Data-Atomic Algorithms for Handl…
Semantically coherent out-of-distribution detection (SCOOD) is a recently proposed realistic OOD detection setting: given labeled in-distribution (ID) data and mixed in-distribution and out-of-distribution unlabeled data as the training…
This paper considers the communication for omniscience (CO) problem: A set of users observe a discrete memoryless multiple source and want to recover the entire multiple source via noise-free broadcast communications. We study the problem…
We consider the problem of constructing an erasure code for storage over a network when the data sources are distributed. Specifically, we assume that there are n storage nodes with limited memory and k<n sources generating the data. We…
Several works have developed vector-linear maximum-distance separable (MDS) storage codes that min- imize the total communication cost required to repair a single coded symbol after an erasure, referred to as repair bandwidth (BW). Vector…
Maximum distance separable (MDS) codes are widely used in distributed storage, but naively repairing a single failure in an $(n,k)$ MDS code requires downloading the full contents of $k$ surviving nodes. Minimum storage regenerating (MSR)…
We consider the problem of multiple-node repair in distributed storage systems under the cooperative model, where the repair bandwidth includes the amount of data exchanged between any two different storage nodes. Recently, explicit…
\emph{Information Dispersal Algorithms (IDAs)} have been widely applied to reliable and secure storage and transmission of data files in distributed systems. An IDA is a method that encodes a file $F$ of size $L=|F|$ into $n$ unrecognizable…
Practical storage systems often adopt erasure codes to tolerate device failures and sector failures, both of which are prevalent in the field. However, traditional erasure codes employ device-level redundancy to protect against sector…
In modern large-scale machine learning applications, the training data are often partitioned and stored on multiple machines. It is customary to employ the "data parallelism" approach, where the aggregated training loss is minimized without…
Placement delivery arrays for distributed computing (Comp-PDAs) have recently been proposed as a framework to construct universal computing schemes for MapReduce-like systems. In this work, we extend this concept to systems with straggling…
Digital contents in large scale distributed storage systems may have different reliability and access delay requirements, and for this reason, erasure codes with different strengths need to be utilized to achieve the best storage…
Distributed storage systems with replication are well known for storing large amount of data. A large number of replication is done in order to provide reliability. This makes the system expensive. Various methods have been proposed over…
Solid-state storage architectures based on NAND or emerging memory devices (SSD), are fundamentally architected and optimized for both reliability and performance. Achieving these simultaneous goals requires co-design of memory components…
High precision atomic data is indispensable for experiments involving studies of fundamental interactions, astrophysics, atomic clocks, plasma science, and others. We develop new parallel atomic structure codes and explore the difficulties…
In a distributed storage system based on erasure coding, an important problem is the \emph{repair problem}: If a node storing a coded piece fails, in order to maintain the same level of reliability, we need to create a new encoded piece and…
The {\em repair locality} of a distributed storage code is the maximum number of nodes that ever needs to be contacted during the repair of a failed node. Having small repair locality is desirable, since it is proportional to the number of…
This paper presents flexible storage codes, a class of error-correcting codes that can recover information from a flexible number of storage nodes. As a result, one can make a better use of the available storage nodes in the presence of…
Regenerating codes provide an efficient way to recover data at failed nodes in distributed storage systems. It has been shown that regenerating codes can be designed to minimize the per-node storage (called MSR) or minimize the…
Cloud storage systems generally add redundancy in storing content files such that $K$ files are replicated or erasure coded and stored on $N > K$ nodes. In addition to providing reliability against failures, the redundant copies can be used…
We derive a lower bound on the amount of information accessed to repair failed nodes within a single rack from any number of helper racks in the rack-aware storage model that allows collective information processing in the nodes that share…