Related papers: Automated data reduction workflows for astronomy
Recent and forthcoming advances in instrumentation, and giant new surveys, are creating astronomical data sets that are not amenable to the methods of analysis familiar to astronomers. Traditional methods are often inadequate not merely…
Robotic automation has transformed scientific workflows in domains such as chemistry and materials science, yet free-space optics, which is a high precision domain, remains largely manual. Optical systems impose strict spatial and angular…
The next generation of telescopes such as the SKA and the Rubin Observatory will produce enormous data sets, requiring automated anomaly detection to enable scientific discovery. Here, we present an overview and friendly user guide to the…
The Production and Distributed Analysis (PanDA) system, originally developed for the ATLAS experiment at the CERN Large Hadron Collider (LHC), has evolved into a robust platform for orchestrating large-scale workflows across distributed…
Optimization of data placement in complex scientific workflows has become very crucial since the large amounts of data generated by these workflows significantly increases the turnaround time of the end-to-end application. It is almost…
We present the AI Cosmologist, an agentic system designed to automate cosmological/astronomical data analysis and machine learning research workflows. This implements a complete pipeline from idea generation to experimental evaluation and…
Industrial recommendation systems (RS) rely on the multi-stage pipeline to balance effectiveness and efficiency when delivering items from a vast corpus to users. Existing RS benchmark datasets primarily focus on the exposure space, where…
Science reproducibility is a cornerstone feature in scientific workflows. In most cases, this has been implemented as a way to exactly reproduce the computational steps taken to reach the final results. While these steps are often…
This document is one of the deliverable reports created for the ESCAPE project. ESCAPE stands for Energy-efficient Scalable Algorithms for Weather Prediction at Exascale. The project develops world-class, extreme-scale computing…
Solar System observations that serve as analogs for exoplanet remote sensing data can provide important opportunities to validate ideas and models related to exoplanet environments. Critically, and unlike true exoplanet observations, Solar…
A lot of work has been done to reach the best possible performance of predictive models on images. There are fewer studies about the resilience of these models when they are trained on image datasets that suffer modifications altering their…
ESPRESSO is an extremely stable high-resolution spectrograph which is currently being developed for the ESO VLT. With its groundbreaking characteristics it is aimed to be a "science machine", i.e. a fully-integrated instrument to directly…
Robo-AO is the first autonomous laser adaptive optics system and science instrument operating on sky. With minimal human oversight, the system robotically executes large scale surveys, monitors long-term astrophysical dynamics and…
We present SIPGI, a spectroscopic pipeline to reduce optical/near-infrared data from slit-based spectrographs. SIPGI is a complete spectroscopic data reduction environment which retains the high level of flexibility and accuracy typical of…
Scientific workflows are widely used to automate scientific data analysis and often involve processing large quantities of data on compute clusters. As such, their execution tends to be long-running and resource intensive, leading to…
The Rapid Telescopes for Optical Response (RAPTOR) experiment is a spatially distributed system of autonomous robotic telescopes that is designed to monitor the sky for optical transients. The core of the system is composed of two telescope…
Designing scientific instrumentation often requires exploring large, highly constrained design spaces using computationally expensive physics simulations. These simulators pose substantial challenges for integrating evolutionary computation…
The operation of instruments and detectors in laboratory or beamline environments presents a complex challenge, requiring stable operation of multiple concurrent devices, often controlled by separate hardware and software solutions. These…
Computational workflow management systems power contemporary data-intensive sciences. The slowly resolving reproducibility crisis presents both a sobering warning and an opportunity to iterate on what science and data processing entails.…
Scientific workflow is a powerful tool to streamline and organize computational steps of scientific application. This paper presents Emerald, a system that adds sophisticated cloud offloading capabilities to scientific workflows. Emerald…