Related papers: Pipeline Collector: gathering performance data for…

The data volumes stored in telescope archives is constantly increasing due to the development and improvements in the instrumentation. Often the archives need to be stored over a distributed storage architecture, provided by independent…

Spectroscopic surveys are undergoing a rapid expansion in their data collecting capabilities, reaching the level of hundreds of spectra per pointing. An efficient use of such huge amounts of information requires a high degree of…

We present the first public version of SImMER, an open-source Python reduction pipeline for astronomical images of point sources. Current capabilities include dark-subtraction, flat-fielding, sky-subtraction, image registration, FWHM…

We present a modular framework, the Workload Characterisation Framework (WCF), that is developed to reproducibly obtain, store and compare key characteristics of radio astronomy processing software. As a demonstration, we discuss the…

New-generation radio telescopes like LOFAR are conducting extensive sky surveys, detecting millions of sources. To maximise the scientific value of these surveys, radio source components must be properly associated into physical sources…

PHOTOMETRYPIPELINE (PP) is an automated pipeline that produces calibrated photometry from imaging data through image registration, aperture photometry, photometric calibration, and target identification with only minimal human interaction.…

lightcurver is a photometric pipeline for time series astronomical imaging data, designed for the semi-automatic extraction of precise light curves from small, blended targets. Such targets include, but are not limited to, lensed quasars,…

In the multi-messenger era, astronomical projects share information about transients phenomena issuing science alerts to the Scientific Community through different communications networks. This coordination is mandatory to understand the…

The Combined Array for Millimeter-wave Astronomy (CARMA) data reduction pipeline (CADRE) has been developed to give investigators a first look at a fully reduced set of their data. It runs automatically on all data produced by the telescope…

A scientific instrument comprised of a global network of millions of independent, connected, remote devices presents unique data acquisition challenges. We describe the software design of a mobile application which collects data from…

The polarization properties of radio sources at very low frequencies (<200 MHz) have not been widely measured, but the new generation of low-frequency radio telescopes, including the Low Frequency Array (LOFAR: a Square Kilometre Array Low…

Distributed Feedback Laser plays a key role as a light source component in optical fiber communication systems ranging from metro, long-haul to submarine one thanks to its competitive features of superior narrow spectral width and…

The pipeline leakage problem is a very challenging and critical issue. Solving this problem will save the nation a lot of money, resources and more importantly, it will save the environment. This paper discusses the state-of-the-art of leak…

Radio pulses emitted in the atmosphere during the air shower development of high-energy primary cosmic rays were measured during the late 1960ies in the frequency range from 2 MHz to 520 MHz. Mainly due to difficulties with radio…

Since October 1, 2006, spectroscopic data from the two FORS instruments have been reduced with a new pipeline, which is based on a bottom-up calibration approach. I give a short description of the pipeline and discuss first experiences with…

We describe the first season COMAP analysis pipeline that converts raw detector readouts to calibrated sky maps. This pipeline implements four main steps: gain calibration, filtering, data selection, and map-making. Absolute gain…

Pulsar studies in the recent years have shown, more than others, to have benefited from a multi-wavelength approach. The INAF - Astronomical Observatory in Cagliari (INAF-OAC) is a growing facility with a young group devoted to pulsar and…

The LOw FRequency ARray - LOFAR is a new radio telescope that is moving the science of radio pulsars and transients into a new phase. Its design places emphasis on digital hardware and flexible software instead of mechanical solutions.…

Modern radio telescopes generate large amounts of data, with the next generation Very Large Array (ngVLA) and the Square Kilometre Array (SKA) expected to feed up to 292 GB of visibilities per second to the science data processor (SDP).…

Streaming, big data applications face challenges in creating scalable data flow pipelines, in which multiple data streams must be collected, stored, queried, and analyzed. These data sources are characterized by their volume (in terms of…