Related papers: Pipeline Collector: gathering performance data for…
The Low Frequency Array (LOFAR) is under construction in the Netherlands and in several surrounding European countries. In this contribution, we describe the layout and design of the telescope, with a particular emphasis on the imaging…
We explore how astronomers take observational data from telescopes, process them into usable scientific data products, curate them for later use, and reuse data for further inquiry. Astronomers have invested heavily in knowledge…
LOFAR (Low Frequency Array) is an innovative radio telescope optimized for the frequency range 30-240 MHz. The telescope is realized as a phased aperture array without any moving parts. Digital beam forming allows the telescope to point to…
We present PyWiFeS, a new Python-based data reduction pipeline for the Wide Field Spectrograph (WiFeS). PyWiFeS consists of a series of core data processing routines built on standard scientific Python packages commonly used in astronomical…
We present the Rubin Observatory system for data storage/retrieval and pipelined code execution. The layer for data storage and retrieval is named the Butler. It consists of a relational database, known as the registry, to keep track of…
The rapid evolution of wearable technologies, such as AR glasses, demands compact, energy-efficient sensors capable of high-precision measurements in dynamic environments. Traditional Frequency-Modulated Continuous Wave (FMCW) Laser…
Powerful detectors at modern experimental facilities routinely collect data at multiple GB/s. Online analysis methods are needed to enable the collection of only interesting subsets of such massive data streams, such as by explicitly…
The pattern of the radio emission of air showers is finely sampled with the Low-Frequency ARray (LOFAR). A set of 382 measured air showers is used to test a fast, analytic parameterization of the distribution of pulse powers. Using this…
The advent of experimental science facilities-instruments and observatories, such as the Large Hadron Collider, the Laser Interferometer Gravitational Wave Observatory, and the upcoming Large Synoptic Survey Telescope-has brought about…
We are performing two complementary pilot pulsar surveys as part of LOFAR commissioning. The LOFAR Pilot Pulsar Survey (LPPS) is a shallow all-sky survey using an incoherent combination of LOFAR stations. The LOFAR Tied-Array Survey (LOTAS)…
The data produced by the future space-based millihertz gravitational-wave detector LISA will require nontrivial pre-processing, which might affect the science results. It is crucial to demonstrate the feasibility of such processing…
As supercomputers continue to grow in scale and capabilities, it is becoming increasingly difficult to isolate processor and system level causes of performance degradation. Over the last several years, a significant number of performance…
Earth observation analytics have the potential to transform many sectors. However, due to limited ground connections, it currently takes hours to days to download and analyze Earth observation data, diminishing the value of data for…
The latest generation of radio astronomy interferometers will conduct all sky surveys with data products consisting of petabytes of spectral line data. Traditional approaches to identifying and parameterising the astrophysical sources…
Data from complex modern astronomical instruments often consist of a large number of different science and calibration files, and their reduction requires a variety of software tools. The execution chain of the tools represents a complex…
Modern network performance monitoring toolkits, such as perfSONAR, take a remarkable number of measurements about the local network environment. To gain a complete picture of network performance, however, one needs to aggregate data across…
This thesis describes the design and implementation of several instruments for digitizing and processing analogue astronomical signals collected using radio telescopes. Modern radio telescopes have significant digital signal processing…
The number of main-sequence stars for which we can observe solar-like oscillations is expected to increase considerably with the short-cadence high-precision photometric observations from the NASA Kepler satellite. Because of this increase…
The Rubin Observatory's Data Butler is designed to allow data file location and file formats to be abstracted away from the people writing the science pipeline algorithms. The Butler works in conjunction with the workflow graph builder to…
The analyst effort in data cleaning is gradually shifting away from the design of hand-written scripts to building and tuning complex pipelines of automated data cleaning libraries. Hyper-parameter tuning for data cleaning is very different…