The Brain Tumor Segmentation (BraTS-METS) Challenge 2023: Brain Metastasis Segmentation on Pre-treatment MRI
Abstract
The translation of AI-generated brain metastases (BM) segmentation into clinical practice relies heavily on diverse, high-quality annotated medical imaging datasets. The BraTS-METS 2023 challenge has gained momentum for testing and benchmarking algorithms using rigorously annotated internationally compiled real-world datasets. This study presents the results of the segmentation challenge and characterizes the challenging cases that impacted the performance of the winning algorithms. Untreated brain metastases on standard anatomic MRI sequences (T1, T2, FLAIR, T1PG) from eight contributed international datasets were annotated in stepwise method: published UNET algorithms, student, neuroradiologist, final approver neuroradiologist. Segmentations were ranked based on lesion-wise Dice and Hausdorff distance (HD95) scores. False positives (FP) and false negatives (FN) were rigorously penalized, receiving a score of 0 for Dice and a fixed penalty of 374 for HD95. Eight datasets comprising 1303 studies were annotated, with 402 studies (3076 lesions) released on Synapse as publicly available datasets to challenge competitors. Additionally, 31 studies (139 lesions) were held out for validation, and 59 studies (218 lesions) were used for testing. Segmentation accuracy was measured as rank across subjects, with the winning team achieving a LesionWise mean score of 7.9. Common errors among the leading teams included false negatives for small lesions and misregistration of masks in space.The BraTS-METS 2023 challenge successfully curated well-annotated, diverse datasets and identified common errors, facilitating the translation of BM segmentation across varied clinical environments and providing personalized volumetric reports to patients undergoing BM treatment.
Keywords
Cite
@article{arxiv.2306.00838,
title = {The Brain Tumor Segmentation (BraTS-METS) Challenge 2023: Brain Metastasis Segmentation on Pre-treatment MRI},
author = {Ahmed W. Moawad and Anastasia Janas and Ujjwal Baid and Divya Ramakrishnan and Rachit Saluja and Nader Ashraf and Nazanin Maleki and Leon Jekel and Nikolay Yordanov and Pascal Fehringer and Athanasios Gkampenis and Raisa Amiruddin and Amirreza Manteghinejad and Maruf Adewole and Jake Albrecht and Udunna Anazodo and Sanjay Aneja and Syed Muhammad Anwar and Timothy Bergquist and Veronica Chiang and Verena Chung and Gian Marco Conte and Farouk Dako and James Eddy and Ivan Ezhov and Nastaran Khalili and Keyvan Farahani and Juan Eugenio Iglesias and Zhifan Jiang and Elaine Johanson and Anahita Fathi Kazerooni and Florian Kofler and Kiril Krantchev and Dominic LaBella and Koen Van Leemput and Hongwei Bran Li and Marius George Linguraru and Xinyang Liu and Zeke Meier and Bjoern H Menze and Harrison Moy and Klara Osenberg and Marie Piraud and Zachary Reitman and Russell Takeshi Shinohara and Chunhao Wang and Benedikt Wiestler and Walter Wiggins and Umber Shafique and Klara Willms and Arman Avesta and Khaled Bousabarah and Satrajit Chakrabarty and Nicolo Gennaro and Wolfgang Holler and Manpreet Kaur and Pamela LaMontagne and MingDe Lin and Jan Lost and Daniel S. Marcus and Ryan Maresca and Sarah Merkaj and Gabriel Cassinelli Pedersen and Marc von Reppert and Aristeidis Sotiras and Oleg Teytelboym and Niklas Tillmans and Malte Westerhoff and Ayda Youssef and Devon Godfrey and Scott Floyd and Andreas Rauschecker and Javier Villanueva-Meyer and Irada Pfluger and Jaeyoung Cho and Martin Bendszus and Gianluca Brugnara and Justin Cramer and Gloria J. Guzman Perez-Carillo and Derek R. Johnson and Anthony Kam and Benjamin Yin Ming Kwan and Lillian Lai and Neil U. Lall and Fatima Memon and Mark Krycia and Satya Narayana Patro and Bojan Petrovic and Tiffany Y. So and Gerard Thompson and Lei Wu and E. Brooke Schrickel and Anu Bansal and Frederik Barkhof and Cristina Besada and Sammy Chu and Jason Druzgal and Alexandru Dusoi and Luciano Farage and Fabricio Feltrin and Amy Fong and Steve H. Fung and R. Ian Gray and Ichiro Ikuta and Michael Iv and Alida A. Postma and Amit Mahajan and David Joyner and Chase Krumpelman and Laurent Letourneau-Guillon and Christie M. Lincoln and Mate E. Maros and Elka Miller and Fanny Moron and Esther A. Nimchinsky and Ozkan Ozsarlak and Uresh Patel and Saurabh Rohatgi and Atin Saha and Anousheh Sayah and Eric D. Schwartz and Robert Shih and Mark S. Shiroishi and Juan E. Small and Manoj Tanwar and Jewels Valerie and Brent D. Weinberg and Matthew L. White and Robert Young and Vahe M. Zohrabian and Aynur Azizova and Melanie Maria Theresa Bruseler and Mohanad Ghonim and Mohamed Ghonim and Abdullah Okar and Luca Pasquini and Yasaman Sharifi and Gagandeep Singh and Nico Sollmann and Theodora Soumala and Mahsa Taherzadeh and Philipp Vollmuth and Martha Foltyn-Dumitru and Ajay Malhotra and Aly H. Abayazeed and Francesco Dellepiane and Philipp Lohmann and Victor M. Perez-Garcia and Hesham Elhalawani and Maria Correia de Verdier and Sanaria Al-Rubaiey and Rui Duarte Armindo and Kholod Ashraf and Moamen M. Asla and Mohamed Badawy and Jeroen Bisschop and Nima Broomand Lomer and Jan Bukatz and Jim Chen and Petra Cimflova and Felix Corr and Alexis Crawley and Lisa Deptula and Tasneem Elakhdar and Islam H. Shawali and Shahriar Faghani and Alexandra Frick and Vaibhav Gulati and Muhammad Ammar Haider and Fatima Hierro and Rasmus Holmboe Dahl and Sarah Maria Jacobs and Kuang-chun Jim Hsieh and Sedat G. Kandemirli and Katharina Kersting and Laura Kida and Sofia Kollia and Ioannis Koukoulithras and Xiao Li and Ahmed Abouelatta and Aya Mansour and Ruxandra-Catrinel Maria-Zamfirescu and Marcela Marsiglia and Yohana Sarahi Mateo-Camacho and Mark McArthur and Olivia McDonnell and Maire McHugh and Mana Moassefi and Samah Mostafa Morsi and Alexander Munteanu and Khanak K. Nandolia and Syed Raza Naqvi and Yalda Nikanpour and Mostafa Alnoury and Abdullah Mohamed Aly Nouh and Francesca Pappafava and Markand D. Patel and Samantha Petrucci and Eric Rawie and Scott Raymond and Borna Roohani and Sadeq Sabouhi and Laura M. Sanchez-Garcia and Zoe Shaked and Pokhraj P. Suthar and Talissa Altes and Edvin Isufi and Yaseen Dhemesh and Jaime Gass and Jonathan Thacker and Abdul Rahman Tarabishy and Benjamin Turner and Sebastiano Vacca and George K. Vilanilam and Daniel Warren and David Weiss and Fikadu Worede and Sara Yousry and Wondwossen Lerebo and Alejandro Aristizabal and Alexandros Karargyris and Hasan Kassem and Sarthak Pati and Micah Sheller and Katherine E. Link and Evan Calabrese and Nourel hoda Tahon and Ayman Nada and Yuri S. Velichko and Spyridon Bakas and Jeffrey D. Rudie and Mariam Aboian},
journal= {arXiv preprint arXiv:2306.00838},
year = {2024}
}