Related papers: Data intensive physics analysis in Azure cloud

In this paper we describe the design, and implementation of the Open Science Data Cloud, or OSDC. The goal of the OSDC is to provide petabyte-scale data cloud infrastructure and related services for scientists working with large quantities…

Scientific research increasingly depends on robust and scalable IT infrastructures to support complex computational workflows. With the proliferation of services provided by research infrastructures, NRENs, and commercial cloud providers,…

The efficient exploitation of worldwide distributed storage and computing resources available in the grids require a robust, transparent and fast deployment of experiment specific software. The approach followed by the CMS experiment at…

A dedicated cosmic muon Monte-Carlo event generator CMSCGEN has been developed for the CMS experiment. The simulation relies on parameterisations of the muon energy and the incidence angle, based on measured and simulated data of the cosmic…

Historically, high energy physics computing has been performed on large purpose-built computing systems. These began as single-site compute facilities, but have evolved into the distributed computing grids used today. Recently, there has…

From particle identification to the discovery of the Higgs boson, deep learning algorithms have become an increasingly important tool for data analysis at the Large Hadron Collider (LHC). We present an innovative end-to-end deep learning…

Computing demands for large scientific experiments, such as the CMS experiment at the CERN LHC, will increase dramatically in the next decades. To complement the future performance increases of software running on central processing units…

The CERN IT provides a set of Hadoop clusters featuring more than 5 PBytes of raw storage with different open-source, user-level tools available for analytical purposes. The CMS experiment started collecting a large set of computing…

In 2010, the Compact Muon Solenoid (CMS) experiment at LHC recorded over 45 pb^-1 of pp collision data at sqrt(s)=7 TeV. The large collected datasets are of very high quality and have been used to commission and calibrate the CMS detector,…

In the era of data-driven science, conducting computational experiments that involve analysing large datasets using heterogeneous computational clusters, is part of the everyday routine for many scientists. Moreover, to ensure the…

The Data Quality Monitoring (DQM) of the Compact Muon Solenoid (CMS) silicon tracking detectors (Tracker) at the Large Hadron Collider (LHC) at CERN is a software based system designed to monitor the detector and reconstruction performance,…

The Compact Muon Solenoid (CMS) Experiment is a general purpose particle detector experiment located at the Large Hadron Collider (LHC) at CERN. In 2008, the LHC beam was commissioned and successfully steered through the CMS detector. First…

We describe the development of a scientific cloud computing (SCC) platform that offers high performance computation capability. The platform consists of a scientific virtual machine prototype containing a UNIX operating system and several…

We present a scalable, cloud-based science platform solution designed to enable next-to-the-data analyses of terabyte-scale astronomical tabular datasets. The presented platform is built on Amazon Web Services (over Kubernetes and S3…

Commercial Cloud computing is becoming mainstream, with funding agencies moving beyond prototyping and starting to fund production campaigns, too. An important aspect of any scientific computing production campaign is data movement, both…

After a brief overview of the Compact Muon Solenoid (CMS) experiment, the status of construction and installation is described in the first part of the note. The second part of the document is devoted to a discussion of the general…

We present a case study of a cloud-based computational workflow for processing large astronomical data sets from the Murchison Widefield Array (MWA) cosmology experiment. Cloud computing is well-suited to large-scale, episodic computation…

Modern cloud has turned data services into easily accessible commodities. With just a few clicks, users are now able to access a catalog of data processing systems for a wide range of tasks. However, the cloud brings in both complexity and…

Scientific computing often requires the availability of a massive number of computers for performing large scale experiments. Traditionally, these needs have been addressed by using high-performance computing solutions and installed…

The data volumes stored in telescope archives is constantly increasing due to the development and improvements in the instrumentation. Often the archives need to be stored over a distributed storage architecture, provided by independent…