FAIR Data Pipeline: provenance-driven data management for traceable scientific workflows
Abstract
Modern epidemiological analyses to understand and combat the spread of disease depend critically on access to, and use of, data. Rapidly evolving data, such as data streams changing during a disease outbreak, are particularly challenging. Data management is further complicated by data being imprecisely identified when used. Public trust in policy decisions resulting from such analyses is easily damaged and is often low, with cynicism arising where claims of "following the science" are made without accompanying evidence. Tracing the provenance of such decisions back through open software to primary data would clarify this evidence, enhancing the transparency of the decision-making process. Here, we demonstrate a Findable, Accessible, Interoperable and Reusable (FAIR) data pipeline developed during the COVID-19 pandemic that allows easy annotation of data as they are consumed by analyses, while tracing the provenance of scientific outputs back through the analytical source code to data sources. Such a tool provides a mechanism for the public, and fellow scientists, to better assess the trust that should be placed in scientific evidence, while allowing scientists to support policy-makers in openly justifying their decisions. We believe that tools such as this should be promoted for use across all areas of policy-facing research.
Cite
@article{arxiv.2110.07117,
title = {FAIR Data Pipeline: provenance-driven data management for traceable scientific workflows},
author = {Sonia Natalie Mitchell and Andrew Lahiff and Nathan Cummings and Jonathan Hollocombe and Bram Boskamp and Ryan Field and Dennis Reddyhoff and Kristian Zarebski and Antony Wilson and Bruno Viola and Martin Burke and Blair Archibald and Paul Bessell and Richard Blackwell and Lisa A Boden and Alys Brett and Sam Brett and Ruth Dundas and Jessica Enright and Alejandra N. Gonzalez-Beltran and Claire Harris and Ian Hinder and Christopher David Hughes and Martin Knight and Vino Mano and Ciaran McMonagle and Dominic Mellor and Sibylle Mohr and Glenn Marion and Louise Matthews and Iain J. McKendrick and Christopher Mark Pooley and Thibaud Porphyre and Aaron Reeves and Edward Townsend and Robert Turner and Jeremy Walton and Richard Reeve},
journal= {arXiv preprint arXiv:2110.07117},
year = {2022}
}