Related papers: The POOL Data Storage, Cache and Conversion Mechan…
Trusting simulation output is crucial for Sandia's mission objectives. We rely on these simulations to perform our high-consequence mission tasks given national treaty obligations. Other science and modeling applications, while they may…
Nowadays, scientific databases have become the bread-and-butter of particle physicists. These databases must be maintained and checked repeatedly to insure the accuracy of their content. The COMPETE collaboration aims at motivating data…
The data volumes stored in telescope archives is constantly increasing due to the development and improvements in the instrumentation. Often the archives need to be stored over a distributed storage architecture, provided by independent…
Data lakes are becoming increasingly prevalent for big data management and data analytics. In contrast to traditional 'schema-on-write' approaches such as data warehouses, data lakes are repositories storing raw data in its original formats…
This proceeding covers tools and technologies at our disposal for scientific data preservation and shows that this extends the scientific reach of our experiments. It is cost-efficient to warehouse data from completed experiments on the…
The CERN Large Hadron Collider (LHC) is designed to collide proton beams of unprecedented energy, in order to extend the frontiers of high-energy particle physics. During the first very successful running period in 2010--2013, the LHC was…
In the context of energy transition, industrial plants that heavily rely on electricity face more and more price volatility. To continue operating in these conditions, the directors become continually more willing to increase their…
This technical note describes the FluidFlower concept, a new laboratory infrastructure for geological carbon storage research. The highly controlled and adjustable system produces a strikingly visual physical ground truth of studied…
Cloud data lakes provide a modern solution for managing large volumes of data. The fundamental principle behind these systems is the separation of compute and storage layers. In this architecture, inexpensive cloud storage is utilized for…
The application of the flow tracing method to power flows in and out of storage units allows to analyse the usage of this technology option in large-scale interconnected electricity systems. We apply this method to a data-driven model of…
Pooling PCIe devices across multiple hosts offers a promising solution to mitigate stranded I/O resources, enhance device utilization, address device failures, and reduce total cost of ownership. The only viable option today are PCIe…
Organizations use data lakes to store and analyze sensitive data. But hackers may compromise data lake storage to bypass access controls and access sensitive data. To address this, we propose Membrane, a system that (1) cryptographically…
During the years 2000 and 2001 the HERA machine and the H1 experiment performed substantial luminosity upgrades. To cope with the increased demands on data handling an effort was made to redesign and modernize the analysis software. Main…
The start of data taking at the Large Hadron Collider will herald a new era in data volumes and distributed processing in particle physics. Data volumes of hundreds of Terabytes will be shipped to Tier-2 centres for analysis by the LHC…
Understanding the fundamental relationships between physics and its information-processing capability has been an active research topic for many years. Physical reservoir computing is a recently introduced framework that allows one to…
Over the last two decades, ROOT TTree has been used for storing over one exabyte of High-Energy Physics (HEP) events. The TTree columnar on-disk layout has been proved to be ideal for analyses of HEP data that typically require access to…
One of the major performance and scalability bottlenecks in large scientific applications is parallel reading and writing to supercomputer I/O systems. The usage of parallel file systems and consistency requirements of POSIX, that all the…
The accumulation of a large amount of new experimental data at an impressive rate at present and future collider experiments has led to important questions concerning data storage and organization, their public access and usability, as well…
ROOT is high energy physics' software for storing and mining data in a statistically sound way, to publish results with scientific graphics. It is evolving since 25 years, now providing the storage format for more than one exabyte of data;…
Emerging high-performance storage technologies are opening up the possibility of designing new distributed data acquisition system architectures, in which the live acquisition of data and their processing are decoupled through a storage…