Related papers: Processing of GASKAP-HI pilot survey data using a …

We present a high-performance, graphics processing unit (GPU)-based framework for the efficient analysis and visualization of (nearly) terabyte (TB)-sized 3-dimensional images. Using a cluster of 96 GPUs, we demonstrate for a 0.5 TB image:…

The Murchison Widefield Array (MWA) is a next-generation radio telescope currently under construction in the remote Western Australia Outback. Raw data will be generated continuously at 5GiB/s, grouped into 8s cadences. This high throughput…

Optimizing data-intensive workflow execution is essential to many modern scientific projects such as the Square Kilometre Array (SKA), which will be the largest radio telescope in the world, collecting terabytes of data per second for the…

The new wide-field radio telescopes, such as: ASKAP, MWA, LOFAR, eVLA and SKA; will produce spectral-imaging data-cubes (SIDC) of unprecedented size -- in the order of hundreds of Petabytes. Servicing such data as images to the end-user in…

This work explores the use of big data technologies deployed in the cloud for processing of astronomical data. We have applied Hadoop and Spark to the task of co-adding astronomical images. We compared the overhead and execution time of…

A new architecture is presented for a Networked Signal Processing System (NSPS) suitable for handling the real-time signal processing of multi-element radio telescopes. In this system, a multi-element radio telescope is viewed as an…

{Context}. The HIFI instrument on the Herschel Space Observatory performed over 9100 astronomical observations, almost 900 of which were calibration observations in the course of the nearly four-year Herschel mission. The data from each…

The previous generation of astronomical instruments tended to consist of single receivers in the focal point of one or more physical reflectors. Because of this, most astronomical data sets were small enough that the raw data could easily…

As software systems increase in complexity, conventional monitoring methods struggle to provide a comprehensive overview or identify performance issues, often missing unexpected problems. Observability, however, offers a holistic approach,…

The paper presents the third data release of Hyper Suprime-Cam Subaru Strategic Program (HSC-SSP), a wide-field multi-band imaging survey with the Subaru 8.2m telescope. HSC-SSP has three survey layers (Wide, Deep, and UltraDeep) with…

In high-resolution solar physics, the volume and complexity of photometric, spectroscopic, and polarimetric ground-based data significantly increased in the last decade reaching data acquisition rates of terabytes per hour. This is driven…

A revolution in radio receiving technology is underway with the development of densely packed phased arrays for radio astronomy. This technology can provide an exceptionally large field of view, while at the same time sampling the sky with…

Innovative developments in data processing, archiving, analysis, and visualization are nowadays unavoidable to deal with the data deluge expected in next-generation facilities for radio astronomy, such as the Square Kilometre Array (SKA)…

The Herschel Common Science System (HCSS) (Ott et al. 2006) (Ott & Science Ground Segment Consortium 2010) is a substantial Java software package, accompanying the development of the Herschel Mission (Pilbratt et al. 2010), supporting all…

Visualisation and analysis of terabyte-scale datacubes, as will be produced with the Australian Square Kilometre Array Pathfinder (ASKAP), will pose challenges for existing astronomy software and the work practices of astronomers. Focusing…

Data management is a critical component of modern experimental workflows. As data generation rates increase, transferring data from acquisition servers to processing servers via conventional file-based methods is becoming increasingly…

We show that, in addition to specific science goals, there is a strong case for conducting an all-sky (i.e. the visible 3-pi steradians) SKA continuum survey which does not fit neatly into conventional science cases. History shows that the…

A major challenge in modern radio astronomy is dealing with the massive data volumes generated by wide-bandwidth receivers. Such massive data rates are often too great for a single device to cope, and so processing must be split across…

The Square Kilometre Array (SKA) will produce spectroscopic surveys of tens to hundreds of millions of HI galaxies, eventually covering 30,000 sq. deg. and reaching out to redshift z~2. The huge volumes probed by the SKA will allow for some…

Particle physics has an ambitious and broad global experimental programme for the coming decades. Large investments in building new facilities are already underway or under consideration. Scaling the present processing power and data…