Related papers: The e-MERLIN Data Reduction Pipeline
The production of photometric light curves from astronomical images is a very time-consuming task. Larger data sets improve the resolution of the light curve, however, the time requirement scales with data volume. The data analysis is often…
The Mapping Nearby Galaxies at Apache Point Observatory (MaNGA) survey, one of three core programs of the fourth-generation Sloan Digital Sky Survey (SDSS-IV), is producing a massive, high-dimensional integral field spectroscopic data set.…
We present a convenient, all-in-one framework for the scientific analysis of fully reduced, (integral-field) spectroscopic data. The GIST pipeline (Galaxy IFU Spectroscopy Tool) is entirely written in Python3 and conducts all steps from the…
Language models of code have demonstrated state-of-the-art performance across various software engineering and source code analysis tasks. However, their demanding computational resource requirements and consequential environmental…
We describe an assembly of numerical tools to model the output data of the Planck satellite. These start with the generation of a CMB sky in a chosen cosmology, add in various foreground sources, convolve the sky signal with arbitrary, even…
The Euclid satellite is an ESA mission that was launched in July 2023. \Euclid is working in its regular observing mode with the target of observing an area of $14\,000~\text{deg}^2$ with two instruments, the Visible Camera (VIS) and the…
The growing demand for machine learning in healthcare requires processing increasingly large electronic health record (EHR) datasets, but existing pipelines are not computationally efficient or scalable. In this paper, we introduce…
We present a technique to automatically minimise the re-computation when a data processing program is iteratively changed, or added to, as is often the case in exploratory data analysis in radio astronomy. A typical example is flagging and…
We present ELSA, a new modular software package, written in C, to analyze and manage spectroscopic data from emission-line objects. In addition to calculating plasma diagnostics and abundances from nebular emission lines, the software…
The era of multi-messenger astrophysics requires rapid and efficient follow-up of transient events, many of which, such as gravitational waves (GW), gamma-ray bursts (GRB), and high-energy neutrinos, suffer from poor sky localisation. We…
Despite the growing availability of Electronic Health Record (EHR) data, researchers often face substantial barriers in effectively using these data for translational research due to their complexity, heterogeneity, and lack of standardized…
Machine Learning (ML) is increasingly used to automate impactful decisions, which leads to concerns regarding their correctness, reliability, and fairness. We envision highly-automated software platforms to assist data scientists with…
We describe the data processing pipeline developed to reduce the pointing observation data of Lunar-based Ultraviolet Telescope (LUT), which belongs to the Chang'e-3 mission of the Chinese Lunar Exploration Program. The pointing observation…
{Context}. The HIFI instrument on the Herschel Space Observatory performed over 9100 astronomical observations, almost 900 of which were calibration observations in the course of the nearly four-year Herschel mission. The data from each…
The Array Camera for Optical to Near-IR Spectrophotometry, or ARCONS, is a camera based on Microwave Kinetic Inductance Detectors (MKIDs), a new technology that has the potential for broad application in astronomy. Using an array of MKIDs,…
The Pan-STARRS Data Processing System is responsible for the steps needed to downloaded, archive, and process all images obtained by the Pan-STARRS telescopes, including real-time detection of transient sources such as supernovae and moving…
We describe a new software package for simulating channelised, high-time resolution data streams from radio telescopes. The software simulates data from the telescope and observing system taking into account the observation strategy,…
The availability of both structured and unstructured databases, such as electronic health data, social media data, patent data, and surveys that are often updated in real time, among others, has grown rapidly over the past decade. With this…
The extension of on-board data processing capabilities is an attractive option to reduce telemetry for scientific instruments on deep space missions. The challenges that this presents, however, require a comprehensive software system, which…
We present the Collection of Elemental Routines for Echelle Spectra (CERES). These routines were developed for the construction of automated pipelines for the reduction, extraction and analysis of spectra acquired with different…