Monte Carlo dropout increases model repeatability
Abstract
The integration of artificial intelligence into clinical workflows requires reliable and robust models. Among the main features of robustness is repeatability. Much attention is given to classification performance without assessing the model repeatability, leading to the development of models that turn out to be unusable in practice. In this work, we evaluate the repeatability of four model types on images from the same patient that were acquired during the same visit. We study the performance of binary, multi-class, ordinal, and regression models on three medical image analysis tasks: cervical cancer screening, breast density estimation, and retinopathy of prematurity classification. Moreover, we assess the impact of sampling Monte Carlo dropout predictions at test time on classification performance and repeatability. Leveraging Monte Carlo predictions significantly increased repeatability for all tasks on the binary, multi-class, and ordinal models leading to an average reduction of the 95% limits of agreement by 17% points.
Cite
@article{arxiv.2111.06754,
title = {Monte Carlo dropout increases model repeatability},
author = {Andreanne Lemay and Katharina Hoebel and Christopher P. Bridge and Didem Egemen and Ana Cecilia Rodriguez and Mark Schiffman and John Peter Campbell and Jayashree Kalpathy-Cramer},
journal= {arXiv preprint arXiv:2111.06754},
year = {2021}
}
Comments
Machine Learning for Health (ML4H) at NeurIPS 2021 - Extended Abstract