We implemented a real-time data processor (rta-dp) framework that can be used to develop real-time analysis pipelines and data handling systems to manage high-throughput data streams with distributed applications in the context of ground and space astrophysical projects and high-energy instruments. The rta-dp is based on the ZeroMQ in-memory communication framework to receive input data, share data between distributed processes, and send or receive commands and pipeline configuration. The rta-dp framework has a flexible architecture that allows the implementation of distributed analysis systems customized to the requirements of several scenarios. The rta-dp framework also provides monitoring capabilities for the running processes and sends housekeeping, logging, alarms, and informative messages that a monitoring process can acquire. We are using the rta-dp in several contexts, such as acquiring and processing data from X-ray detectors to the data quality system of the ASTRI Project, as well as reprocessing and archiving data.
@article{arxiv.2511.03760,
title = {The Real-Time Data Processor Framework for Data Handling and Analysis of High-Energy Instruments},
author = {A. Bulgarelli and N. Parmiggiani and L. Castaldini and R. Falco and A. Di Piano and V. Fioretti and G. Panebianco and A. Rizzo},
journal= {arXiv preprint arXiv:2511.03760},
year = {2025}
}
Comments
4 pages, 2 figures, proceeding of Astronomical Data Analysis Software and System XXXIV (2024), to appear in the ASP conference series