Related papers: Big Data in HEP: A comprehensive use case study
This white paper briefly summarized key conclusions of the recent US Community Study on the Future of Particle Physics (Snowmass 2021) workshop on Software and Computing for Small High Energy Physics Experiments.
HEP community leads and operates cutting-edge experiments for the DOE Office of Science which have challenging sensing, data processing, and computing requirements that far surpass typical industrial applications. To make necessary progress…
These recommendations are the result of reflections by scientists and experts who are, or have been, involved in the preservation of high-energy physics data. The work has been done under the umbrella of the Data Lifecycle panel of the…
With the steady increase in the precision of flavour physics measurements collected during LHC Run 2, the LHCb experiment requires simulated data samples of larger and larger sizes to study the detector response in detail. The simulation of…
Colliders, among the most successful tools of particle physics, have revealed much about matter. This review describes how colliders contribute to the search for particle dark matter, focusing on the highest-energy collider currently in…
At large scales, quantum systems may become advantageous over their classical counterparts at performing certain tasks. Developing tools to analyse these systems at the relevant scales, in a manner consistent with quantum mechanics, is…
High-Performance Computing (HPC) platforms enable scientific software to achieve breakthroughs in many research fields such as physics, biology, and chemistry, by employing Research Software Engineering (RSE) techniques. These include 1)…
In high energy physics (HEP) experiments, the reconstruction of charged particle trajectories is one of the most fundamental yet computationally expensive parts of event processing. At future hadron colliders such as the High-Luminosity…
Science Data Systems (SDS) handle science data from acquisition through processing to distribution. They are deployed in the Cloud today, and the efficiency of Cloud instance utilization is critical to success. Conventional SDS are unable…
The H1 data preservation project was started in 2009 as part of the global data preservation initiative in high-energy physics, DPHEP. In order to retain the full potential for future improvements, the H1 Collaboration aims for level 4 of…
High performance computing (HPC) has driven collaborative science discovery for decades. Exascale computing platforms, currently in the design stage, will be deployed around 2022. The next generation of supercomputers is expected to utilize…
The HL-LHC run is anticipated to start at the end of this decade and will pose a significant challenge for the scale of the HEP software and computing infrastructure. The mission of the U.S. CMS Software & Computing Operations Program is to…
In this chapter of the High Energy Physics Software Foundation Community Whitepaper, we discuss the current state of infrastructure, best practices, and ongoing developments in the area of data and software preservation in high energy…
Over the last two decades, ROOT TTree has been used for storing over one exabyte of High-Energy Physics (HEP) events. The TTree columnar on-disk layout has been proved to be ideal for analyses of HEP data that typically require access to…
High energy physics data is a long term investment and contains the potential for physics results beyond the lifetime of a collaboration. Many existing experiments are concluding their physics programs, and looking at ways to preserve their…
The application of machine learning (ML) in high energy physics (HEP), specifically in heavy-flavor jet tagging at Large Hadron Collider (LHC) experiments, has experienced remarkable growth and innovation in the past decade. This review…
To extract physics results from the recorded data, the LHC experiments are using Grid computing infrastructure. The event data processing on the Grid requires scalable access to non-event data (detector conditions, calibrations, etc.)…
The Big Data management is a problem right now. The Big Data growth is very high. It is very difficult to manage due to various characteristics. This manuscript focuses on Big Data analytics in cloud environment using Hadoop. We have…
Within the next decade, experimental High Energy Physics (HEP) will enter a new era of scientific discovery through a set of targeted programs recommended by the Particle Physics Project Prioritization Panel (P5), including the upcoming…
With the rapid growth of Next Generation Sequencing (NGS) technologies, large amounts of "omics" data are daily collected and need to be processed. Indexing and compressing large sequences datasets are some of the most important tasks in…