Related papers: Adaptive Data Reduction Workflows for Astronomy --…
We present the data reduction software and the distribution of Level 1 and Level 2 products of the Stratospheric Terahertz Observatory 2 (STO2). STO2, a balloon-borne Terahertz telescope, surveyed star-forming regions and the Galactic plane…
EUSO-SPB2 (Extreme Universe Space Observatory on a Super Pressure Balloon II) is a precursor mission for a future space observatory for multi-messenger astrophysics, planned to be launched in Spring 2023 with a flight duration target of 100…
The ATLAS Event Streaming Service (ESS) at the LHC is an approach to preprocess and deliver data for Event Service (ES) that has implemented a fine-grained approach for ATLAS event processing. The ESS allows one to asynchronously deliver…
We describe the data processing pipeline developed to reduce the pointing observation data of Lunar-based Ultraviolet Telescope (LUT), which belongs to the Chang'e-3 mission of the Chinese Lunar Exploration Program. The pointing observation…
Dimensionality reduction can be applied to hyperspectral images so that the most useful data can be extracted and processed more quickly. This is critical in any situation in which data volume exceeds the capacity of the computational…
ScopeSim is a flexible multipurpose instrument data simulation framework built in Python. It enables both raw and reduced observation data to be simulated for a wide range of telescopes and instruments quickly and efficiently on a personal…
In recent years, there has been a growing demand for improved autonomy for in-orbit operations such as rendezvous, docking, and proximity maneuvers, leading to increased interest in employing Deep Learning-based Spacecraft Pose Estimation…
Pulsar data analysis pipelines have historically been comprised of bespoke software systems, supporting the off-line analysis of data. However modern data acquisition systems are making off-line analyses impractical. They often output…
Modern astronomical data processing requires complex software pipelines to process ever growing datasets. For radio astronomy, these pipelines have become so large that they need to be distributed across a computational cluster. This makes…
We developed a Python based framework for astronomical image processing and analysis. Astronomical image loading, normalizing, stacking, and filtering processes represent visible range images from grayscale. Besides, the blending process…
Compared to traditional distributed computing environments such as grids, cloud computing provides a more cost-effective way to deploy scientific workflows. Each task of a scientific workflow requires several large datasets that are located…
To overcome the energy and bandwidth limitations of traditional IoT systems, edge computing or information extraction at the sensor node has become popular. However, now it is important to create very low energy information extraction or…
The operation-planning of satellites, aimed at introducing a certain level of supervised automation during the execution of the operations, poses a great challenge to both designers and operators. From one side, the routine operations for…
The DSN Transient Observatory (DTO) is a signal processing facility that can monitor up to four DSN downlink bands for astronomically interesting signals. The monitoring is done commensally with reception of deep space mission telemetry.…
Scientific workflow management systems offer features for composing complex computational pipelines from modular building blocks, for executing the resulting automated workflows, and for recording the provenance of data products resulting…
A systematic pipeline for data processing and knowledge discovery is essential to extracting knowledge from big data and making recommendations for operational decision-making. The CRISP-DM model is the de-facto standard for developing…
SDAMS is the ensemble of database + software packages aimed to the archiving, quick-look analysis, off-line analysis, network accessibility and plotting of the SPOrt produced data. Many of the aspects related to data archiving, analysis and…
We present a fully modular and scalable software pipeline for processing electron microscope (EM) images of brain slices into 3D visualization of individual neurons and demonstrate an end-to-end segmentation of a large EM volume using a…
Performing data-intensive analytics is an essential part of modern Earth science. As such, research in atmospheric physics and meteorology frequently requires the processing of very large observational and/or modeled datasets. Typically,…
This paper presents some of the main aspects of the software library that has been developed for the reduction of optical and infrared images, an integral part of the end-to-end survey system being built to support public imaging surveys at…