Related papers: The Data Processing Pipeline for the Herschel-HIFI…
Simulations with high accuracy are an essential part of scientific research to accelerate the innovation process. They are especially useful for finding novel approaches or optimizing existing methods. Today, powerful software tools are…
In high-energy particle physics, workflow management systems are primarily used as tailored solutions in dedicated areas such as Monte Carlo production. However, physicists performing data analyses are usually required to steer their…
The VST Telescope Control Software logs continuously detailed information about the telescope and instrument operations. Commands, telemetries, errors, weather conditions and anything may be relevant for the instrument maintenance and the…
We present a fully modular and scalable software pipeline for processing electron microscope (EM) images of brain slices into 3D visualization of individual neurons and demonstrate an end-to-end segmentation of a large EM volume using a…
We describe an assembly of numerical tools to model the output data of the Planck satellite. These start with the generation of a CMB sky in a chosen cosmology, add in various foreground sources, convolve the sky signal with arbitrary, even…
We present a convenient, all-in-one framework for the scientific analysis of fully reduced, (integral-field) spectroscopic data. The GIST pipeline (Galaxy IFU Spectroscopy Tool) is entirely written in Python3 and conducts all steps from the…
This paper presents the High Frequency Instrument (HFI) data processing procedures for the Planck 2018 release. Major improvements in mapmaking have been achieved since the previous 2015 release. They enabled the first significant…
In this paper we describe an unmanned aerial system equipped with a thermal-infrared camera and software pipeline that we have developed to monitor animal populations for conservation purposes. Taking a multi-disciplinary approach to tackle…
A wealth of cosmological and astrophysical information is expected from many ongoing and upcoming large-scale surveys. It is crucial to prepare for these surveys now and develop tools that can efficiently extract most information. We…
The article provides a brief description of the software package DECH for processing and analysis of astronomical spectra. DECH supports all stages of processing and analysis of spectral data, including image preprocessing, spectra…
Youpi stands for "YOUpi is your processing PIpeline". It is a portable, easy to use web application providing high level functionalities to perform data reduction on scientific FITS images. It is built on top of open source processing tools…
Earth observation analytics have the potential to transform many sectors. However, due to limited ground connections, it currently takes hours to days to download and analyze Earth observation data, diminishing the value of data for…
The volume of data generated by modern astronomical telescopes is extremely large and rapidly growing. However, current high-performance data processing architectures/frameworks are not well suited for astronomers because of their…
Computer running human machine interfaces are fast supplanting conventional hardware dedicated to monitoring and supervising tasks. This kind of approach was successfully employed to develop a new monitoring and control software, running on…
Automated code instrumentation, i.e. the insertion of measurement hooks into a target application by the compiler, is an established technique for collecting reliable, fine-grained performance data. The set of functions to instrument has to…
Data analysis in space sciences has been performed exclusively visually for years, despite the fact that the largest amount of data belongs to non-visible portions of the electromagnetic spectrum. This, on the one hand, limits the study of…
Over the last two decades, the field of computational science has seen a dramatic shift towards incorporating high-throughput computation and big-data analysis as fundamental pillars of the scientific discovery process. This has…
There are numerous approaches to building analysis applications across the high-energy physics community. Among them are Python-based, or at least Python-driven, analysis workflows. We aim to ease the adoption of a Python-based analysis…
We present here a provenance management system adapted to astronomical projects needs. We collected use cases from various astronomy projects and defined a data model in the ecosystem developed by the IVOA (International Virtual Observatory…
This work explores the use of big data technologies deployed in the cloud for processing of astronomical data. We have applied Hadoop and Spark to the task of co-adding astronomical images. We compared the overhead and execution time of…