Related papers: Distributed data storage for modern astroparticle …
Today, the operating TAIGA (Tunka Advanced Instrument for cosmic rays and Gamma Astronomy) experiment continuously produces and accumulates a large volume of raw astroparticle data. To be available for the scientific community these data…
The XRootD system is used to transfer, store, and cache large datasets from high-energy physics (HEP). In this study we focus on its capability as distributed on-demand storage cache. Through exploring a large set of daily log files between…
The heterogeneous edge-cloud computing paradigm can provide a more optimal direction to deploy scientific workflows than traditional distributed computing or cloud computing environments. Due to the different sizes of scientific datasets…
A common task in scientific computing is the derivation of data. This workflow extracts the most important information from large input data and stores it in smaller derived data objects. The derived data objects can then be used for…
In the era of data-driven science, conducting computational experiments that involve analysing large datasets using heterogeneous computational clusters, is part of the everyday routine for many scientists. Moreover, to ensure the…
This work presents novel distributed data collection systems and storage algorithms for collaborative learning wireless sensor networks (WSNs). In a large WSN, consider $n$ collaborative sensor devices distributed randomly to acquire…
Experiment design has a rich history dating back over a century and has found many critical applications across various fields since then. The use and collection of users' data in experiments often involve sensitive personal information, so…
Conceptually exoplanet research has one foot in the discipline of Astrophysics and the other foot in Planetary Science. Research strategies for exoplanets will require efficient access to data and information from both realms. Astrophysics…
Huge amount of data with both space and text information, e.g., geo-tagged tweets, is flooding on the Internet. Such spatio-textual data stream contains valuable information for millions of users with various interests on different keywords…
Dispatchability of renewable energy sources and inflexible loads can be achieved using a volatility-compensating energy storage. However, as the future power outputs of the inflexible devices are uncertain, the computation of a dispatch…
In many sciences the increasing amounts of data are reaching the limit of established data handling and processing. With four large research centers of the German Helmholtz association the Large Scale Data Management and Analysis (LSDMA)…
With the rapid development of mobile computing technology, massive amounts of spatial data are continuously generated from various mobile terminals and sensing devices, such as smartphones, connected vehicles, and drones. Performing…
With the increasing computational power of current supercomputers, the size of data produced by scientific simulations is rapidly growing. To reduce the storage footprint and facilitate scalable post-hoc analyses of such scientific data…
We present DASPack, a high-performance, open-source compression tool specifically designed for distributed acoustic sensing (DAS) data. As DAS becomes a key technology for real-time, high-density, and long-range monitoring in fields such as…
Cloud computing provides scientists a platform that can deploy computation and data intensive applications without infrastructure investment. With excessive cloud resources and a decision support system, large generated data sets can be…
Data-intensive platforms such as Hadoop and Spark are routinely used to process massive amounts of data residing on distributed file systems like HDFS. Increasing memory sizes and new hardware technologies (e.g., NVRAM, SSDs) have recently…
The heterogeneous edge-cloud computing paradigm can provide an optimal solution to deploy scientific workflows compared to cloud computing or other traditional distributed computing environments. Owing to the different sizes of scientific…
Data on transient events, like GRBs, are often contained in large databases of unstructured data from space experiments, merged with potentially large amount of background or simply undesired information. We present a computational formal…
More and more distributed software systems are being developed and deployed today. Like other software, distributed software systems also need very strong quality assurance support. Distributed software is often very large/complex, has…
This paper presents DESTinE Block, a blockchain-based data storage framework designed for power systems and optimized for resource-constrained environments, including grid-edge devices such as single-board computers. The proposed…