The Medical Segmentation Decathlon
Abstract
International challenges have become the de facto standard for comparative assessment of image analysis algorithms given a specific task. Segmentation is so far the most widely investigated medical image processing task, but the various segmentation challenges have typically been organized in isolation, such that algorithm development was driven by the need to tackle a single specific clinical problem. We hypothesized that a method capable of performing well on multiple tasks will generalize well to a previously unseen task and potentially outperform a custom-designed solution. To investigate the hypothesis, we organized the Medical Segmentation Decathlon (MSD) - a biomedical image analysis challenge, in which algorithms compete in a multitude of both tasks and modalities. The underlying data set was designed to explore the axis of difficulties typically encountered when dealing with medical images, such as small data sets, unbalanced labels, multi-site data and small objects. The MSD challenge confirmed that algorithms with a consistent good performance on a set of tasks preserved their good average performance on a different set of previously unseen tasks. Moreover, by monitoring the MSD winner for two years, we found that this algorithm continued generalizing well to a wide range of other clinical problems, further confirming our hypothesis. Three main conclusions can be drawn from this study: (1) state-of-the-art image segmentation algorithms are mature, accurate, and generalize well when retrained on unseen tasks; (2) consistent algorithmic performance across multiple tasks is a strong surrogate of algorithmic generalizability; (3) the training of accurate AI segmentation models is now commoditized to non AI experts.
Cite
@article{arxiv.2106.05735,
title = {The Medical Segmentation Decathlon},
author = {Michela Antonelli and Annika Reinke and Spyridon Bakas and Keyvan Farahani and AnnetteKopp-Schneider and Bennett A. Landman and Geert Litjens and Bjoern Menze and Olaf Ronneberger and Ronald M. Summers and Bram van Ginneken and Michel Bilello and Patrick Bilic and Patrick F. Christ and Richard K. G. Do and Marc J. Gollub and Stephan H. Heckers and Henkjan Huisman and William R. Jarnagin and Maureen K. McHugo and Sandy Napel and Jennifer S. Goli Pernicka and Kawal Rhode and Catalina Tobon-Gomez and Eugene Vorontsov and Henkjan Huisman and James A. Meakin and Sebastien Ourselin and Manuel Wiesenfarth and Pablo Arbelaez and Byeonguk Bae and Sihong Chen and Laura Daza and Jianjiang Feng and Baochun He and Fabian Isensee and Yuanfeng Ji and Fucang Jia and Namkug Kim and Ildoo Kim and Dorit Merhof and Akshay Pai and Beomhee Park and Mathias Perslev and Ramin Rezaiifar and Oliver Rippel and Ignacio Sarasua and Wei Shen and Jaemin Son and Christian Wachinger and Liansheng Wang and Yan Wang and Yingda Xia and Daguang Xu and Zhanwei Xu and Yefeng Zheng and Amber L. Simpson and Lena Maier-Hein and M. Jorge Cardoso},
journal= {arXiv preprint arXiv:2106.05735},
year = {2022}
}