The BRAVO Semantic Segmentation Challenge Results in UNCV2024
Abstract
We propose the unified BRAVO challenge to benchmark the reliability of semantic segmentation models under realistic perturbations and unknown out-of-distribution (OOD) scenarios. We define two categories of reliability: (1) semantic reliability, which reflects the model's accuracy and calibration when exposed to various perturbations; and (2) OOD reliability, which measures the model's ability to detect object classes that are unknown during training. The challenge attracted nearly 100 submissions from international teams representing notable research institutions. The results reveal interesting insights into the importance of large-scale pre-training and minimal architectural design in developing robust and reliable semantic segmentation models.
Cite
@article{arxiv.2409.15107,
title = {The BRAVO Semantic Segmentation Challenge Results in UNCV2024},
author = {Tuan-Hung Vu and Eduardo Valle and Andrei Bursuc and Tommie Kerssies and Daan de Geus and Gijs Dubbelman and Long Qian and Bingke Zhu and Yingying Chen and Ming Tang and Jinqiao Wang and Tomáš Vojíř and Jan Šochman and Jiří Matas and Michael Smith and Frank Ferrie and Shamik Basu and Christos Sakaridis and Luc Van Gool},
journal= {arXiv preprint arXiv:2409.15107},
year = {2024}
}
Comments
ECCV 2024 proceeding paper of the BRAVO challenge 2024, see https://benchmarks.elsa-ai.eu/?ch=1&com=introduction Corrected numbers in Tables 1,3,4,5 and 10