English
Related papers

Related papers: Distributed storage algorithms with optimal tradeo…

200 papers

Randomized exponential backoff is a widely deployed technique for coordinating access to a shared resource. A good backoff protocol should, arguably, satisfy three natural properties: (i) it should provide constant throughput, wasting as…

Distributed, Parallel, and Cluster Computing · Computer Science 2015-07-14 Michael A. Bender , Jeremy T. Fineman , Seth Gilbert , Maxwell Young

Most of today's distributed machine learning systems assume {\em reliable networks}: whenever two machines exchange information (e.g., gradients or models), the network should guarantee the delivery of the message. At the same time, recent…

Distributed, Parallel, and Cluster Computing · Computer Science 2019-05-17 Chen Yu , Hanlin Tang , Cedric Renggli , Simon Kassing , Ankit Singla , Dan Alistarh , Ce Zhang , Ji Liu

We analyze the convergence of gradient-based optimization algorithms that base their updates on delayed stochastic gradient information. The main application of our results is to the development of gradient-based distributed optimization…

Optimization and Control · Mathematics 2011-05-02 Alekh Agarwal , John C. Duchi

The increasing interest in serverless computation and ubiquitous wireless networks has led to numerous connected devices in our surroundings. Among such devices, IoT devices have access to an abundance of raw data, but their inadequate…

Distributed, Parallel, and Cluster Computing · Computer Science 2021-04-12 Ramyad Hadidi , Jiashen Cao , Hyesoon Kim

In cache-aided networks, the server populates the cache memories at the users during low-traffic periods, in order to reduce the delivery load during peak-traffic hours. In turn, there exists a fundamental trade-off between the delivery…

Information Theory · Computer Science 2019-03-15 Abdelrahman M. Ibrahim , Ahmed A. Zewail , Aylin Yener

In this paper a stochastic model of a large distributed system where users' files are duplicated on unreliable data servers is investigated. Due to a server breakdown, a copy of a file can be lost, it can be retrieved if another copy of the…

Probability · Mathematics 2015-12-21 Wen Sun , Mathieu Feuillet , Philippe Robert

Regenerating codes are a class of recently developed codes for distributed storage that, like Reed-Solomon codes, permit data recovery from any subset of k nodes within the n-node network. However, regenerating codes possess in addition,…

Information Theory · Computer Science 2015-03-17 Nihar B. Shah , K. V. Rashmi , P. Vijay Kumar , Kannan Ramchandran

Motivated by the need for robust and fast distributed computation in highly dynamic Peer-to-Peer (P2P) networks, we study algorithms for the fundamental distributed agreement problem. P2P networks are highly dynamic networks that experience…

Data Structures and Algorithms · Computer Science 2015-03-19 John Augustine , Gopal Pandurangan , Peter Robinson , Eli Upfal

In coding for distributed storage systems, efficient data reconstruction and repair through accessing a predefined number of arbitrarily chosen storage nodes is guaranteed by regenerating codes. Traditionally, code parameters, specially the…

Information Theory · Computer Science 2018-01-01 Kaveh Mahdaviani , Soheil Mohajer , Ashish Khisti

Many modern schedulers can dynamically adjust their service capacity to match the incoming workload. At the same time, however, unpredictability and instability in service capacity often incur operational and infrastructure costs. In this…

Optimization and Control · Mathematics 2020-05-12 Yorie Nakahira , Andres Ferragut , Adam Wierman

A consistency/latency tradeoff arises as soon as a distributed storage system replicates data. For low latency, modern storage systems often settle for weak consistency conditions, which provide little, or even worse, no guarantee for data…

Distributed, Parallel, and Cluster Computing · Computer Science 2015-07-16 Hengfeng Wei , Yu Huang , Jiannong Cao , Jian Lu

We consider the setting of distributed storage system where a single file is subdivided into smaller fragments of same size which are then replicated with a common replication factor across servers of identical cache size. An incoming file…

Information Theory · Computer Science 2022-06-27 Rooji Jinan , Ajay Badita , Pradeep Sarvepalli , Parimal Parag

Consistent hashing is a technique for distributing data across a network of nodes in a way that minimizes reorganization when nodes join or leave the network. It is extensively applied in modern distributed systems as a fundamental…

Distributed, Parallel, and Cluster Computing · Computer Science 2025-03-18 Massimo Coluzzi , Amos Brocco , Alessandro Antonucci , Tiziano Leidi

Network cache allocation and management are important aspects of the design of an Information-Centric Network (ICN), such as one based on Named Data Networking (NDN). We address the problem of optimal cache size allocation and content…

Optimization and Control · Mathematics 2021-05-13 Van Sy Mai , Stratis Ioannidis , Davide Pesavento , Lotfi Benmohamed

Erasure coding is a storage-efficient alternative to replication for achieving reliable data backup in distributed storage systems. During the storage process, traditional erasure codes require a unique source node to create and upload all…

Distributed, Parallel, and Cluster Computing · Computer Science 2015-03-19 Lluis Pamies-Juarez , Anwitaman Datta , Frédérique Oggier

Due to individual unreliable commodity components, failures are common in large-scale distributed storage systems. Erasure codes are widely deployed in practical storage systems to provide fault tolerance with low storage overhead. However,…

Distributed, Parallel, and Cluster Computing · Computer Science 2020-04-15 Liangliang Xu , Min Lyu , Zhipeng Li , Yongkun Li , Yinlong Xu

Data availability is critical in distributed storage systems, especially when node failures are prevalent in real life. A key requirement is to minimize the amount of data transferred among nodes when recovering the lost or unavailable data…

Distributed, Parallel, and Cluster Computing · Computer Science 2013-06-07 Runhui Li , Jian Lin , Patrick P. C. Lee

In wireless cloud storage systems, the recovery failure probability depends on not only wireless channel conditions but also storage size of each distributed storage node. For an efficient utilization of limited storage capacity and the…

Information Theory · Computer Science 2016-11-17 Bi Hong , Wan Choi

Social storage systems are a good alternative to existing data backup systems of local, centralized, and P2P backup. In this paper, we look at two untouched aspects of social storage systems. One aspect involves modelling social storage as…

Computer Science and Game Theory · Computer Science 2018-04-11 Pramod Mane , Kapil Ahuja , Nagarajan Krishnamurthy

We consider the problem of managing a dynamic heterogeneous storage system in a distributed way so that the amount of data assigned to a host in that system is related to its capacity. Two central problems have to be solved for this: (1)…

Distributed, Parallel, and Cluster Computing · Computer Science 2013-07-26 Sebastian Kniesburges , Andreas Koutsopoulos , Christian Scheideler