Related papers: TeraScale SneakerNet: Using Inexpensive Disks for …
The SKA will be capable of producing a stream of science data products that are Exa-scale in terms of their storage and processing requirements. This Google-scale enterprise is attracting considerable international interest and excitement…
Many industries rely on visual insights to support decision- making processes in their businesses. In mining, the analysis of drills and geological shapes, represented as 3D geometries, is an important tool to assist geologists on the…
Apache Spark is a Big Data framework for working on large distributed datasets. Although widely used in the industry, it remains rather limited in the academic community or often restricted to software engineers. The goal of this paper is…
The analysis and an efficient scientific exploration of the Digital Palomar Observatory Sky Survey (DPOSS) represents a major technical challenge. The input data set consists of 3 Terabytes of pixel information, and contains a few billion…
We describe a distributed processing cluster of inexpensive Linux machines developed jointly by the Astronomy and Computer Science departments at Haverford College which has been successfully used to search a large volume of data from a…
K-nearest neighbor search is one of the fundamental tasks in various applications and the hierarchical navigable small world (HNSW) has recently drawn attention in large-scale cloud services, as it easily scales up the database while…
Distributed ledger technology such as blockchain is considered essential for supporting large numbers of micro-transactions in the Machine Economy, which is envisioned to involve billions of connected heterogeneous and decentralized…
ISP minimizes data transfer for analytics but faces challenges in adaptation and disaggregation. We propose DockerSSD, an ISP model leveraging OS-level virtualization and lightweight firmware to enable containerized data processing directly…
We propose to develop a new method of information storage to replace magnetic hard disk drives and other instruments of secondary/backup data storage. The proposed method stores petabytes of user-data in a sugar cube (1 cm3), and can…
Cloud computing is a complex infrastructure of software, hardware, processing, and storage that is available as a service. Cloud computing offers immediate access to large numbers of the world's most sophisticated supercomputers and their…
The abundance of available sensor and derived data from large scientific experiments, such as earth observation programs, radio astronomy sky surveys, and high-energy physics already exceeds the storage hardware globally fabricated per…
As conventional storage density reaches its physical limits, the cost of a gigabyte of storage is no longer plummeting, but rather has remained mostly flat for the past decade. Meanwhile, file sizes continue to grow, leading to ever fuller…
The Sloan Digital Sky Survey science database is approaching 2TB. While the vast majority of queries normally execute in seconds or minutes, this interactive execution time can be disproportionately increased by a small fraction of queries…
Despite the growing need for data of more and more sophisticated 3D reconstruction pipelines, we can still observe a scarcity of suitable public datasets. Existing 3D datasets are either low resolution, limited to a small amount of scenes,…
The possibility of using an IBM PC-compatible computer as an NFS file server for a moderate-size workgroup has been explored. A pilot project currently under way at LBNL has proven that this platform does indeed provide a viable alternative…
The CXL-based solid-state drive (CXL-SSD) provides a promising approach towards scaling the main memory capacity at low cost. However, the CXL-SSD faces performance challenges due to the long flash access latency and unpredictable events…
This paper considers distributed storage systems (DSSs) from a graph theoretic perspective. A DSS is constructed by means of the path decomposition of a 3- regular graph into P4 paths. The paths represent the disks of the DSS and the edges…
This paper proposes a novel transmission strategy, referred to as layered BPSK, which allows two independent symbol streams to be layered at the transmitter on a non-orthogonal basis and isolated from each other at the receiver without…
Digital planetariums can provide a broader range of educational experiences than the more classical planetariums that use star-balls. This is because of their ability to project images, content from current research and the 3D distribution…
Distributed Ledger Technologies (DLT) and Decentralized File Storages (DFS) are becoming increasingly used to create common, decentralized and trustless infrastructures where participants interact and collaborate in Peer-to-Peer…