Related papers: The MKID Science Data Pipeline
KIDSpec, the Kinetic Inductance Detector Spectrometer, is a proposed optical to near IR Microwave Kinetic Inductance Detector (MKID) spectrograph. MKIDs are superconducting photon counting detectors which are able to resolve the energy of…
Data scientists develop ML pipelines in an iterative manner: they repeatedly screen a pipeline for potential issues, debug it, and then revise and improve its code according to their findings. However, this manual process is tedious and…
The last 15 years have seen a surge of new multi-chip optical and near-IR imagers. While some of them are accompanied by specific reduction pipelines, user-friendly and generic reduction tools are uncommon. In this paper I introduce THELI,…
In this work, a multi-stage Machine Learning (ML) pipeline is proposed for pipe leakage detection in an industrial environment. As opposed to other industrial and urban environments, the environment under study includes many interfering…
The BULLKID-DM experiment aims to detect WIMP-like potential Dark Matter particles with masses below 1 GeV/c^2. Sensing these particles is challenging, as it requires nuclear recoil detectors characterized by high exposure and an energy…
The Stratospheric Kinetic Inductance Polarimeter (SKIP) is a proposed balloon-borne experiment designed to study the cosmic microwave background, the cosmic infrared background and Galactic dust emission by observing 1133 square degrees of…
We describe the goals, design, implementation, and initial progress of the UKIRT Infrared Deep Sky Survey (UKIDSS), a seven year sky survey which began in May 2005, using the UKIRT Wide Field Camera. It is a portfolio of five survey…
Miriad is a radio interferometry data-reduction package, designed for taking raw data through to the image analysis stage. The Miriad project, begun in 1988, is now middle-aged. With the wisdom of hindsight, we review design decisions and…
In this paper, we report on the construction of a new and independent pipeline for analyzing the public data from the first observing run of advanced LIGO for mergers of compact binary systems. The pipeline incorporates different techniques…
The Euclid satellite is an ESA mission that was launched in July 2023. \Euclid is working in its regular observing mode with the target of observing an area of $14\,000~\text{deg}^2$ with two instruments, the Visible Camera (VIS) and the…
Building large, cryogenic MKID arrays requires processing highly-multiplexed, wideband readout signals in real time; a task that has previously required large, heavy, and power-intensive custom electronics. In this work, we present the…
This paper defines the UKIRT Infrared Deep Sky Survey (UKIDSS) Early Data Release (EDR). UKIDSS is a set of five large near-infra-red surveys defined by Lawrence et al. (2006), being undertaken with the UK Infra-red Telescope (UKIRT) Wide…
This paper describes a machine learning and data science pipeline for structured information extraction from documents, implemented as a suite of open-source tools and extensions to existing tools. It centers around a methodology for…
With the arrival of a number of wide-field snapshot image-plane radio transient surveys, there will be a huge influx of images in the coming years making it impossible to manually analyse the datasets. Automated pipelines to process the…
We argue for the need for a new generation of data science solutions that can democratize recent advances in data engineering and artificial intelligence for non-technical users from various disciplines, enabling them to unlock the full…
Millimeter wave communication systems can leverage information from sensors to reduce the overhead associated with link configuration. LIDAR (light detection and ranging) is one sensor widely used in autonomous driving for high resolution…
This work describes the architecture and implementation of a high-data-rate, energy-efficient millimeter-wave (mm-wave) waveguide solution for integration inside an Integrated Circuit (IC) package. The complete waveguide solution (together…
The input data pipeline is an essential component of each machine learning (ML) training job. It is responsible for reading massive amounts of training data, processing batches of samples using complex transformations, and loading them onto…
BlueMUSE is an integral field spectrograph in an early development stage for the ESO VLT. For our design of the data reduction software for this instrument, we are first reviewing capabilities and issues of the pipeline of the existing MUSE…
In recent years, a wide variety of automated machine learning (AutoML) methods have been proposed to search and generate end-to-end learning pipelines. While these techniques facilitate the creation of models for real-world applications,…