Related papers: Data reduction pipeline for the MMT Magellan Infra…

With the arrival of a number of wide-field snapshot image-plane radio transient surveys, there will be a huge influx of images in the coming years making it impossible to manually analyse the datasets. Automated pipelines to process the…

We present a novel compression algorithm for reducing the storage of LiDAR sensor data streams. Our model exploits spatio-temporal relationships across multiple LiDAR sweeps to reduce the bitrate of both geometry and intensity values.…

We developed a high-speed image reduction pipeline using Graphics Processing Units (GPUs) as hardware accelerators. Astronomers desire detecting EM counterpart of gravitational-wave sources as soon as possible for sharing positional…

Data scientists develop ML pipelines in an iterative manner: they repeatedly screen a pipeline for potential issues, debug it, and then revise and improve its code according to their findings. However, this manual process is tedious and…

A novel deep learning-based magnetic resonance imaging reconstruction pipeline was designed to address fundamental image quality limitations of conventional reconstruction to provide high-resolution, low-noise MR images. This pipeline's…

As a dedicated solar radio interferometer, the MingantU SpEctral RadioHeliograph (MUSER) generates massive observational data in the frequency range of 400 MHz -- 15 GHz. High-performance imaging forms a significantly important aspect of…

The Son-Of-XShooter (SOXS) is a single object spectrograph (UV-VIS & NIR) and acquisition camera scheduled to be mounted on the ESO 3.58-m New Technology Telescope at the La Silla Observatory. Although the underlying data reduction…

Automated machine learning (AutoML) aims for constructing machine learning (ML) pipelines automatically. Many studies have investigated efficient methods for algorithm selection and hyperparameter optimization. However, methods for ML…

One of the most important changes in observational astronomy of the 21st Century is a rapid shift from classical object-by-object observations to extensive automatic surveys. As CCD detectors are getting better and their prices are getting…

Data preprocessing pipelines, which includes data decoding, cleaning, and transforming, are a crucial component of Machine Learning (ML) training. Thy are computationally intensive and often become a major bottleneck, due to the increasing…

Executing machine learning (ML) pipelines in real-time on radiology images is hard due to the limited computing resources in clinical environments and the lack of efficient data transfer capabilities to run them on research clusters. We…

SOFIA presents a number of interesting challenges for the development of a data reduction environment which, at its initial phase, will have to incorporate pipelines from seven different instruments. Therefore, the SOFIA data reduction…

We present our image processing system for the reduction of optical imaging data from multi-chip cameras. In the framework of the Garching Bonn Deep Survey (GaBoDS; Schirmer et al. 2003) consisting of about 20 square degrees of high-quality…

We present the Spitzer/IRS Mapping Legacy Archive (SIMLA); a complete set of mid-infrared spectral cubes built from low-resolution mapping-mode fixed-target observations from Spitzer/IRS (5.2-38 micron, R~60-130). Contained in this dataset…

Novel multiplexing triple-axis neutron scattering spectrometers yield significant improvements of the common triple-axis instruments. While the planar scattering geometry keeps ensuring compatibility with complex sample environments, a…

Currently, a variety of pipeline tools are available for use in data engineering. Data scientists can use these tools to resolve data wrangling issues associated with data and accomplish some data engineering tasks from data ingestion…

We have reduced the data taken with the Spectral and Photometric Imaging Receiver (SPIRE) photometer on board the Herschel Space Observatory in the Science Demonstration Phase (SDP) of the Herschel Astrophysical Terahertz Large Area Survey…

The direct detection and characterization of planetary and substellar companions at small angular separations is a rapidly advancing field. Dedicated high-contrast imaging instruments deliver unprecedented sensitivity, enabling detailed…

MUSE, a 2nd generation VLT instrument, will become the world's largest integral field spectrograph. It will be an AO assisted instrument which, in a single exposure, covers the wavelength range from 465 to 930 nm with an average resolution…

In this paper, we describe the SPHEREx image and spectrophotometry data processing pipeline, an infrastructure and software system designed to produce calibrated spectral images and photometric measurements for NASA's SPHEREx mission.…