Automatically Finding and Categorizing Replication Studies
Digital Libraries
2023-11-28 v1 Computation and Language
Abstract
In many fields of experimental science, papers that failed to replicate continue to be cited as a result of the poor discoverability of replication studies. As a first step to creating a system that automatically finds replication studies for a given paper, 334 replication studies and 344 replicated studies were collected. Replication studies could be identified in the dataset based on text content at a higher rate than chance (AUROC = 0.886). Additionally, successful replication studies could be distinguished from failed replication studies at a higher rate than chance (AUROC = 0.664).
Keywords
Cite
@article{arxiv.2311.15055,
title = {Automatically Finding and Categorizing Replication Studies},
author = {Bob de Ruiter},
journal= {arXiv preprint arXiv:2311.15055},
year = {2023}
}