Related papers: Container-Based Pre-Pipeline Data Processing on HP…

The X-Ray Imaging and Spectroscopy Mission (XRISM) is the seventh Japanese X-ray observatory whose development and operation are in collaboration with universities and research institutes in Japan, the United States, and Europe, including…

XRISM (X-Ray Imaging and Spectroscopy Mission), with the Resolve high-resolution spectrometer and the Xtend wide-field imager on-board, is designed to build on the successes of the abbreviated Hitomi mission to address outstanding…

We propose a simulation-based approach for performance modeling of parallel applications on high-performance computing platforms. Our approach enables full-system performance modeling: (1) the hardware platform is represented by an abstract…

The high-performance computing (HPC) community has recently seen a substantial diversification of hardware platforms and their associated programming models. From traditional multicore processors to highly specialized accelerators, vendors…

This work elaborates on a High performance computing (HPC) architecture based on Simple Linux Utility for Resource Management (SLURM) [1] for deploying heterogeneous Large Language Models (LLMs) into a scalable inference engine. Dynamic…

Modern radio telescopes generate large amounts of data, with the next generation Very Large Array (ngVLA) and the Square Kilometre Array (SKA) expected to feed up to 292 GB of visibilities per second to the science data processor (SDP).…

The amount of remote sensing data available to applications is constantly growing due to the rise of very-high-resolution sensors and short repeat cycle satellites. Consequently, tackling computational complexity in Earth Observation…

In this paper, we describe the optical imaging data processing pipeline developed for the Subaru Telescope's Hyper Suprime-Cam (HSC) instrument. The HSC Pipeline builds on the prototype pipeline being developed by the Large Synoptic Survey…

XRISM is an X-ray astronomical mission by the JAXA, NASA, ESA and other international participants, that is planned for launch in 2022 (Japanese fiscal year), to quickly restore high-resolution X-ray spectroscopy of astrophysical objects.…

High-performance computing (HPC) systems and cloud data centers are converging, and containers are becoming the default method of portable software deployment. Yet, while containers simplify software management, they face significant…

Pulsar data analysis pipelines have historically been comprised of bespoke software systems, supporting the off-line analysis of data. However modern data acquisition systems are making off-line analyses impractical. They often output…

The X-Ray Telescope (XRT) onboard the Hinode satellite, launched 23 September 2006 by the Japanese Aerospace Exploration Agency (JAXA) is a joint mission between Japan, the United States, and the United Kingdom to study the solar corona. In…

Cloud computing for high performance computing resources is an emerging topic. This service is of interest to researchers who care about reproducible computing, for software packages with complex installations, and for companies or…

Deploying complex, distributed scientific workflows across diverse HPC sites is often hindered by site-specific dependencies and complex build environments. This paper investigates the design and performance of portable HPC container images…

The Exa.TrkX project has applied geometric learning concepts such as metric learning and graph neural networks to HEP particle tracking. Exa.TrkX's tracking pipeline groups detector measurements to form track candidates and filters them.…

Atmospheric characterisation of exoplanets from the ground is an actively growing field of research. In this context we have created the ATMOSPHERIX consortium: a research project aimed at characterizing exoplanets atmospheres using…

The Herschel Space Observatory is the fourth cornerstone mission in the ESA science programme and performs photometry and spectroscopy in the 55 - 672 micron range. The development of the Herschel Data Processing System started in 2002 to…

ixpeobssim is a simulation and analysis framework, based on the Python programming language and the associated scientific ecosystem, specifically developed for the Imaging X-ray Polarimetry Explorer (IXPE). Given a source model and the…

SAKURAONE is a managed high performance computing (HPC) cluster developed and operated by the SAKURA Internet Research Center. It builds on the KOKARYOKU PHY bare metal GPU platform and is optimized for advanced workloads, including large…

Many scientific workloads are comprised of many tasks, where each task is an independent simulation or analysis of data. The execution of millions of tasks on heterogeneous HPC platforms requires scalable dynamic resource management and…