Related papers: LOFAR and HDF5: Toward a New Radio Data Standard
Transient radio phenomena and pulsars are one of six LOFAR Key Science Projects (KSPs). As part of the Transients KSP, the Pulsar Working Group (PWG) has been developing the LOFAR Pulsar Data Pipelines to both study known pulsars as well as…
Radio astronomy file formats are now required to store wide frequency bandwidths and multiple simultaneous receiver beams and must be able to account for versatile observing modes and numerous calibration strategies. The need to capture and…
The FITS (Flexible Image Transport System) data format was developed in the late 1970s for storage and exchange of astronomy-related image data. Since then, it has become a standard file format not only for images, but also for radio…
The FITS (Flexible Image Transport System) data format has been the de facto data format for astronomy-related data products since its inception in the late 1970s. While the FITS file format is widely supported, it lacks many of the…
With SKA precursor and pathfinder operations in full swing, radio and (sub-)mm astronomy is entering the era of super big data. The big questions is how to make (sub-)mm and radio data available to the astronomical community, preferably…
Modern astronomical data processing requires complex software pipelines to process ever growing datasets. For radio astronomy, these pipelines have become so large that they need to be distributed across a computational cluster. This makes…
A number of hardware upgrades for the Low-Frequency Array (LOFAR) are currently under development. These upgrades are collectively referred to as the LOFAR 2.0 upgrade. The first stage of LOFAR 2.0 will introduce a distributed clock signal…
The Starlink Hierarchical Data System has been a very successful niche astronomy file format and library for over 30 years. Development of the library was frozen ten years ago when funding for Starlink was stopped and almost no-one remains…
LOFAR, the Low Frequency Array, is a next-generation radio telescope that is being built in Northern Europe and expected to be fully operational at the end of this decade. It will operate at frequencies from 15 to 240 MHz (corresponding to…
We propose a new file format named "H5MD" for storing molecular simulation data, such as trajectories of particle positions and velocities, along with thermodynamic observables that are monitored during the course of the simulation. H5MD…
The Low Frequency Array (LOFAR) radio telescope is an international aperture synthesis radio telescope used to study the Universe at low frequencies. One of the goals of the LOFAR telescope is to conduct deep wide-field surveys. Here we…
Radio astronomy is entering the era of large surveys. This paper describes the plans for wide surveys with the LOw Frequency ARray (LOFAR) and their synergy with large surveys at higher frequencies (in particular in the 1-2 GHz band) that…
Raw data sizes are growing and proliferating in scientific research, driven by the success of data-hungry computational methods, such as machine learning. The preponderance of proprietary and shoehorned data formats make computations slower…
Despite almost all being acquired as photons, astronomical data from different instruments and at different stages in its life may exist in different formats to serve different purposes. Beyond the data itself, descriptive information is…
LOFAR, the Low-Frequency Array, is a next-generation software-driven radio telescope operating between 30 and 240MHz, currently under construction by ASTRON in the Netherlands. This low frequency radio band is one of the few largely…
The volume of radio-astronomical data is a considerable burden in the processing and storing of radio observations with high time and frequency resolutions and large bandwidths. Lossy compression of interferometric radio-astronomical data…
The data volumes stored in telescope archives is constantly increasing due to the development and improvements in the instrumentation. Often the archives need to be stored over a distributed storage architecture, provided by independent…
Most major scientific results produced by ground-based gamma-ray telescopes in the last 30 years have been obtained by expert members of the collaborations operating these instruments. This is due to the proprietary data and software…
The Forschungszentrum Juelich has been hosting the German part of the LOFAR archive since 2013. It is Germany's most extensive radio astronomy archive, currently storing nearly 22 petabytes (PB) of data. Future radio telescopes are expected…
The Low Frequency Array (LOFAR) Two-metre Sky Survey (LoTSS) is a low-frequency radio continuum survey of the Northern sky at an unparalleled resolution and sensitivity. In order to fully exploit this huge dataset and those produced by the…