Related papers: Pipeline Collector: gathering performance data for…
Advanced instruments in a variety of scientific domains are collecting massive amounts of data that must be post-processed and organized to support scientific research activities. Astronomers have been pioneers in the use of databases to…
High performance computing has been used in various fields of astrophysical research. But most of it is implemented on massively parallel systems (supercomputers) or graphical processing unit clusters. With the advent of multicore…
LOFAR, the Low-Frequency Array, is a powerful new radio telescope operating between 10 and 240 MHz. LOFAR allows detailed sensitive high-resolution studies of the low-frequency radio sky. At the same time LOFAR also provides excellent short…
A program that we call the QUEST Data Processing Software Pipeline has been written to process the large volumes of data produced by the QUEST camera on the Samuel Oschin Schmidt Telescope at the Palomar Observatory. The program carries out…
Air showers from cosmic rays emit short, intense radio pulses. LOFAR is a new radio telescope, that is being built in the Netherlands and Europe. Designed primarily as a radio interferometer, the core of LOFAR will have a high density of…
A recent development in radio astronomy is to replace traditional dishes with many small antennas. The signals are combined to form one large, virtual telescope. The enormous data streams are cross-correlated to filter out noise. This is…
This paper presents the Container Profiler, a software tool that measures and records the resource usage of any containerized task. Our tool profiles the CPU, memory, disk, and network utilization of containerized tasks collecting over…
LOFAR (the Low Frequency Array), a distributed digital radio telescope with stations in the Netherlands, Germany, France, Sweden, and the United Kingdom, is designed to enable full-sky monitoring of transient radio sources. These…
The Offline software framework for data analysis of the Pierre Auger Observatory is a set of computational tools developed to cater to the needs of a large and geographically dispersed collaboration established to measure the spectrum,…
We discuss the possibilities of measuring ultra-high energy cosmic rays and neutrinos with radio techniques. We review a few of the properties of radio emission from cosmic ray air showers and show how these properties can be explained by…
Given the extreme precision attainable with the Kepler Space Telescope, the mitigation of instrumental artefacts is very important. In an earlier paper (Murphy 2012), the characteristics of Kepler data were discussed in light of their…
PySE is a Python software package for finding and measuring sources in radio telescope images. The software was designed to detect sources in the LOFAR telescope images, but can be used with images from other radio telescopes as well. We…
The AARTFAAC project aims to implement an All-Sky Monitor (ASM), using the Low Frequency Array (LOFAR) telescope. It will enable real-time, 24x7 monitoring for low frequency radio transients over most of the sky locally visible to the LOFAR…
In this paper, we report on the construction of a new and independent pipeline for analyzing the public data from the first observing run of advanced LIGO for mergers of compact binary systems. The pipeline incorporates different techniques…
The LOw-Frequency ARray (LOFAR) is used to make precise measurements of radio emission from extensive air showers, yielding information about the primary cosmic ray. Interpreting the measured data requires an absolute and…
In this paper we describe an unmanned aerial system equipped with a thermal-infrared camera and software pipeline that we have developed to monitor animal populations for conservation purposes. Taking a multi-disciplinary approach to tackle…
The Square Kilometre Array (SKA) will be both the largest radio telescope ever constructed and the largest Big Data project in the known Universe. The first phase of the project will generate on the order of 5 zettabytes of data per year. A…
Reducing a set of numbers to a single value is a fundamental operation in applications such as signal processing, data compression, scientific computing, and neural networks. Accumulation, which involves summing a dataset to obtain a single…
{Context}. The HIFI instrument on the Herschel Space Observatory performed over 9100 astronomical observations, almost 900 of which were calibration observations in the course of the nearly four-year Herschel mission. The data from each…
The challenge to fully exploit the potential of existing and upcoming scientific instruments like large single-dish radio telescopes is to process the collected massive data effectively and efficiently. As a "quasi 2D stencil computation"…