Related papers: Pipeline Collector: gathering performance data for…
One of the science drivers of the new Low Frequency Array (LOFAR) is large-area surveys of the low-frequency radio sky. Realizing this goal requires automated processing of the interferometric data, such that fully calibrated images are…
Radio astronomy is entering a new era with new and future radio observatories such as the Low Frequency Array and the Square Kilometer Array. We describe in detail an automated flagging pipeline and evaluate its performance. With only a…
Transient radio phenomena and pulsars are one of six LOFAR Key Science Projects (KSPs). As part of the Transients KSP, the Pulsar Working Group (PWG) has been developing the LOFAR Pulsar Data Pipelines to both study known pulsars as well as…
Traditional radio telescopes use large, steel dishes to observe radio sources. The LOFAR radio telescope is different, and uses tens of thousands of fixed, non-movable antennas instead, a novel design that promises ground-breaking research…
LOFAR is a leading aperture synthesis telescope operated in the Netherlands with stations across Europe. The LOFAR Two-meter Sky Survey (LoTSS) will produce more than 3000 14 TB data sets, mapping the entire northern sky at low frequencies.…
The Low Frequency Array (LOFAR) radio telescope is an international aperture synthesis radio telescope used to study the Universe at low frequencies. One of the goals of the LOFAR telescope is to conduct deep wide-field surveys. Here we…
Current and future astronomical survey facilities provide a remarkably rich opportunity for transient astronomy, combining unprecedented fields of view with high sensitivity and the ability to access previously unexplored wavelength…
LOFAR is a low-frequency array distributed across several European countries. Each LOFAR station contains thousands of antennas and associated electronics, making monitoring and thorough testing of those components essential to ensuring…
The volume of data generated by modern astronomical telescopes is extremely large and rapidly growing. However, current high-performance data processing architectures/frameworks are not well suited for astronomers because of their…
The HIFI data processing pipeline was developed to systematically process diagnostic, calibration and astronomical observations taken with the HIFI science instrumentas part of the Herschel mission. The HIFI pipeline processed data from all…
Radio astronomy observatories with high throughput back end instruments require real-time data processing. While computing hardware continues to advance rapidly, development of real-time processing pipelines remains difficult and…
Data processing pipelines represent an important slice of the astronomical software library that include chains of processes that transform raw data into valuable information via data reduction and analysis. In this work we present Corral,…
Pulsar data analysis pipelines have historically been comprised of bespoke software systems, supporting the off-line analysis of data. However modern data acquisition systems are making off-line analyses impractical. They often output…
With the arrival of a number of wide-field snapshot image-plane radio transient surveys, there will be a huge influx of images in the coming years making it impossible to manually analyse the datasets. Automated pipelines to process the…
New scientific instruments are starting to generate an unprecedented amount of data. LOFAR, one of the Square Kilometre Array pathfinders, is already producing data on a petabyte scale. The calibration of these data presents a huge…
We present a new Python pipeline for processing data from astronomical long-slit spectroscopy observations recorded with CCD detectors. The pipeline is designed to aim for simplicity, manual execution, transparency and robustness. The goal…
The Low Frequency Array (LOFAR) is an ideal instrument to conduct deep extragalactic surveys. It has a large field of view and is sensitive to large scale and compact emission. It is, however, very challenging to synthesize thermal noise…
LOFAR is a new and sensitive radio interferometer that can be used for dynamic high-resolution imaging spectroscopy at low radio frequencies from 10 to 90 and 110 to 250 MHz. Here we describe its usage for observations of the Sun and in…
This paper presents the DDF Pipeline, a radio astronomy data processing tool initially designed for the LOw-Frequency ARray (LO- FAR) radio-telescope and a candidate for processing data from the Square Kilometre Array (SKA). This work…
We have designed and implemented a novel way to process wide-field astronomical data within a distributed environment of hardware resources and humanpower. The system is characterized by integration of archiving, calibration, and…