A numerical variability approach to results stability tests and its application to neuroimaging
Abstract
Ensuring the long-term reproducibility of data analyses requires results stability tests to verify that analysis results remain within acceptable variation bounds despite inevitable software updates and hardware evolutions. This paper introduces a numerical variability approach for results stability tests, which determines acceptable variation bounds using random rounding of floating-point calculations. By applying the resulting stability test to \fmriprep, a widely-used neuroimaging tool, we show that the test is sensitive enough to detect subtle updates in image processing methods while remaining specific enough to accept numerical variations within a reference version of the application. This result contributes to enhancing the reliability and reproducibility of data analyses by providing a robust and flexible method for stability testing.
Keywords
Cite
@article{arxiv.2307.01373,
title = {A numerical variability approach to results stability tests and its application to neuroimaging},
author = {Yohan Chatelain and Loïc Tetrel and Christopher J. Markiewicz and Mathias Goncalves and Gregory Kiar and Oscar Esteban and Pierre Bellec and Tristan Glatard},
journal= {arXiv preprint arXiv:2307.01373},
year = {2023}
}