Related papers: Billion-files File Systems (BfFS): A Comparison
Creating and destroying threads on modern Linux systems incurs high latency, absent concurrency, and fails to scale as we increase concurrency. To address this concern we introduce a process-local cache of idle threads. Specifically,…
The growing interest in reliable multi-party applications has fostered widespread adoption of Byzantine Fault-Tolerant (BFT) consensus protocols. Existing BFT protocols need f more replicas than Paxos-style protocols to prevent equivocation…
Large Language Models (LLMs) possess a theoretical capability to model information density far beyond the limits of classical statistical methods (e.g., Lempel-Ziv). However, utilizing this capability for lossless compression involves…
High-performance object stores are an emerging technology which offers an alternative solution in the field of HPC storage, with potential to address long-standing scalability issues in traditional distributed POSIX file systems due to…
Breadth-first Search (BFS) is one of the most important graph processing subroutines, especially for computing the unweighted distance. Many applications may require running BFS from multiple sources. Sequentially, when running BFS on a…
In a variety of applications, we need to keep track of the development of a data set over time. For maintaining and querying this multi version data I/O-efficiently, external memory data structures are required. In this paper, we present a…
A wide variety of large-scale data has been produced in bioinformatics. In response, the need for efficient handling of biomedical big data has been partly met by parallel computing. However, the time demand of many bioinformatics programs…
Although every individual invented storage technology made a big step towards perfection, none of them is spotless. Different data store essentials such as performance, availability, and recovery requirements have not met together in a…
The remote procedure call (a.k.a. RPC) latency becomes increasingly significant in a distributed file system. We propose BuffetFS, a user-level file system that optimizes I/O performance by eliminating the RPCs caused by \texttt{open()}…
The use of under-utilized Internet resources is widely recognized as a viable form of high performance computing. Sustained processing power of roughly 40T FLOPS using 4 million volunteered Internet hosts has been reported for…
Existing software-based memory tiering systems decide which pages to place on the slower or faster tier. However, they do not take into account two important factors that greatly influence application performance: the size of the migrated…
This paper presents the Container Profiler, a software tool that measures and records the resource usage of any containerized task. Our tool profiles the CPU, memory, disk, and network utilization of containerized tasks collecting over…
BitTorrent is undoubtedly the most popular P2P file sharing application on today's Internet. The widespread popularity of BitTorrent has attracted a great deal of attention from networking researchers who conducted various performance…
The article provides a new approach to creating hierarchical structure of file system. First, it gives overview of the existing ways of storing files in current operating systems. Second, it describes the new way of building structures of a…
Byzantine-Fault-Tolerant (BFT) systems are rapidly emerging as a viable technology for production-grade systems, notably in closed consortia deployments for nancial and supply-chain applications. Unfortunately, most algorithms proposed so…
The InterPlanetary File System (IPFS) is a peer-to-peer network for storing data in a distributed file system, hosting over 190,000 peers spanning 152 countries. Despite its prominence, the privacy properties that IPFS offers to peers are…
Fixing software faults contributes significantly to the cost of software maintenance and evolution. Techniques for reducing these costs require datasets of software faults, as well as an understanding of the faults, for optimal testing and…
Federated recommender system (FRS), which enables many local devices to train a shared model jointly without transmitting local raw data, has become a prevalent recommendation paradigm with privacy-preserving advantages. However, previous…
Many extreme-scale applications require the movement of large quantities of data to, from, and among leadership computing facilities, as well as other scientific facilities and the home institutions of facility users. These applications,…
Data-intensive applications often require exploratory analysis of large datasets. If analysis is performed on distributed resources, data locality can be crucial to high throughput and performance. We propose a "data diffusion" approach…