Medical imaging is an important research field with many opportunities for improving patients' health. However, there are a number of challenges that are slowing down the progress of the field as a whole, such optimizing for publication. In this paper we reviewed several problems related to choosing datasets, methods, evaluation metrics, and publication strategies. With a review of literature and our own analysis, we show that at every step, potential biases can creep in. On a positive note, we also see that initiatives to counteract these problems are already being started. Finally we provide a broad range of recommendations on how to further these address problems in the future. For reproducibility, data and code for our analyses are available on \url{https://github.com/GaelVaroquaux/ml_med_imaging_failures}
@article{arxiv.2103.10292,
title = {How I failed machine learning in medical imaging -- shortcomings and recommendations},
author = {Gaël Varoquaux and Veronika Cheplygina},
journal= {arXiv preprint arXiv:2103.10292},
year = {2022}
}