Related papers: An Automated Pipeline for the VST Data Log Analysi…
Pulsar data analysis pipelines have historically been comprised of bespoke software systems, supporting the off-line analysis of data. However modern data acquisition systems are making off-line analyses impractical. They often output…
Modern telescope facilities generate data from various sources, including sensors, weather stations, LiDARs, and FRAMs. Sophisticated software architectures using the Internet of Things (IoT) and big data technologies are required to manage…
Time series forecasting is fundamental for various use cases in different domains such as energy systems and economics. Creating a forecasting model for a specific use case requires an iterative and complex design process. The typical…
The production of science-ready data from major solar telescopes requires expertise beyond that of the typical observer. This is a consequence of the increasing complexity of instruments and observing sequences, which require calibrations…
The Large Array Survey Telescope (LAST) is a wide-field telescope designed to explore the variable and transient sky with a high cadence and to be a test-bed for cost-effective telescope design. A LAST node is composed of 48 (32 already…
The photometric calibration of the Sloan Digital Sky Survey (SDSS) is a multi-step process which involves data from three different telescopes: the 1.0-m telescope at the US Naval Observatory (USNO), Flagstaff Station, Arizona (which was…
The VISTA Data Flow System comprises nightly pipeline and archiving of near infrared data from UKIRT-WFCAM and VISTA. This includes multi-epoch data which can be used to find moving and variable objects. We have developed a new model for…
Data scientists develop ML pipelines in an iterative manner: they repeatedly screen a pipeline for potential issues, debug it, and then revise and improve its code according to their findings. However, this manual process is tedious and…
Glitches are the observational manifestations of superfluidity inside neutron stars. The aim of this paper is to describe an automated glitch detection pipeline, which can alert the observers on possible real-time detection of rotational…
The New Vacuum Solar Telescope (NVST) is a 1-meter vacuum solar telescope that aims to observe the fine structures of active regions on the Sun. The main tasks of the NVST are high resolution imaging and spectral observations, including the…
Machine Learning (ML) is increasingly used to automate impactful decisions, which leads to concerns regarding their correctness, reliability, and fairness. We envision highly-automated software platforms to assist data scientists with…
The Chinese Space Station Survey Telescope (CSST) is a flagship space-based observatory. Its main survey camera is designed to conduct high spatial resolution near-ultraviolet to near-infrared imaging and low-resolution spectroscopic…
With the arrival of a number of wide-field snapshot image-plane radio transient surveys, there will be a huge influx of images in the coming years making it impossible to manually analyse the datasets. Automated pipelines to process the…
System logs perform a critical function in software-intensive systems as logs record the state of the system and significant events in the system at important points in time. Unfortunately, log entries are typically created in an ad-hoc,…
The traditional mode of recording faults in heavy factory equipment has been via hand marked inspection sheets, wherein a machine engineer manually marks the faulty machine regions on a paper outline of the machine. Over the years, millions…
In this contribution we present the FAST, which is a comprehensive software suite that aims to streamline and automatically manage the forecast of atmospheric and astroclimatic parameters (provided respectively by Meso-Nh and Astro-Meso-Nh…
In this paper, we describe the optical imaging data processing pipeline developed for the Subaru Telescope's Hyper Suprime-Cam (HSC) instrument. The HSC Pipeline builds on the prototype pipeline being developed by the Large Synoptic Survey…
Log analysis and monitoring are essential aspects in software maintenance and identifying defects. In particular, the temporal nature and vast size of log data leads to an interesting and important research question: How can logs be…
AstroStat is an easy-to-use tool for performing statistical analysis on data. It has been designed to be compatible with Virtual Observatory (VO) standards thus enabling it to become an integral part of the currently available collection of…
The Large Synoptic Survey Telescope (LSST) is an 8-m optical ground-based telescope being constructed on Cerro Pachon in Chile. LSST will survey half the sky every few nights in six optical bands. The data will be transferred to NCSA and…