Related papers: Any Data, Any Time, Anywhere: Global Data Access f…
Science Data Systems (SDS) handle science data from acquisition through processing to distribution. They are deployed in the Cloud today, and the efficiency of Cloud instance utilization is critical to success. Conventional SDS are unable…
Today's astronomical projects need computational systems capable to store and analyze large amounts of scientific data, to effectively share data with other research Institutes and to easily implement information services to present data…
Publicly available data from open sources (e.g., United States Census Bureau (Census), World Health Organization (WHO), Intergovernmental Panel on Climate Change (IPCC)) are vital resources for policy makers, students and researchers across…
The HEP community is approaching an era were the excellent performances of the particle accelerators in delivering collision at high rate will force the experiments to record a large amount of information. The growing size of the datasets…
The rapid growth of Artificial Intelligence and Machine Learning in scientific research has highlighted a gap between industry-standard MLOps tools and platforms, and the unique requirements of modern and Open Science, particularly…
Data is collected everywhere in our increasingly instrumented world and people are increasingly wanting to access this data from anywhere in it. This kind of anywhere & everywhere data present new challenges and opportunities for…
The accumulation of a large amount of new experimental data at an impressive rate at present and future collider experiments has led to important questions concerning data storage and organization, their public access and usability, as well…
The theoretical foundations of a new model and paradigm (called TIE) for data storage and access are introduced. Associations between data elements are stored in a single Matrix table, which is usually kept entirely in RAM for quick access.…
Scientific experiments and modern applications are generating large amounts of data every day. Most organizations utilize In-house servers or Cloud resources to manage application data and workload. The traditional database management…
Influenced by the advances in data and computing, the scientific practice increasingly involves machine learning and artificial intelligence driven methods which requires specialized capabilities at the system-, science- and service-level…
Large High Energy Physics (HEP) experiments adopted a distributed computing model more than a decade ago. WLCG, the global computing infrastructure for LHC, in partnership with the US Open Science Grid, has achieved data management at the…
The CMS (Compact Muon Solenoid) experiment is one of the two large general-purpose particle physics detectors built at the LHC (Large Hadron Collider) at CERN in Geneva, Switzerland. The diverse collaboration combined with a highly…
Data from high-energy physics (HEP) experiments are collected with significant financial and human effort and are in many cases unique. At the same time, HEP has no coherent strategy for data preservation and re-use, and many important and…
For the past decade, HENP experiments have been heading towards a distributed computing model in an effort to concurrently process tasks over enormous data sets that have been increasing in size as a function of time. In order to optimize…
This is a personal perspective on data sharing in the context of public data releases suitable for generic analysis. These open data can be a powerful tool for expanding the science of high energy physics, but care must be taken in when,…
All Control Systems that grow to any size have a variety of data that are stored in different formats on different nodes in the network. Examples include sensor value and status, archived sensor data, device oriented support data and…
The continuous growth of data production in almost all scientific areas raises new problems in data access and management, especially in a scenario where the end-users, as well as the resources that they can access, are worldwide…
Scientific workflows have become highly heterogenous, leveraging distributed facilities such as High Performance Computing (HPC), Artificial Intelligence (AI), Machine Learning (ML), scientific instruments (data-driven pipelines) and edge…
ESA Gaia mission is producing the more accurate source catalogue in astronomy up to now. That represents a challenge on the archiving area to make accessible this information to the astronomers in an efficient way. Also, new astronomical…
Exploratory data analysis tools must respond quickly to a user's questions, so that the answer to one question (e.g. a visualized histogram or fit) can influence the next. In some SQL-based query systems used in industry, even very large…