Do deep learning models for instance segmentation generalize to novel objects in a systematic way? For classification, such behavior has been questioned. In this study, we aim to understand if certain design decisions such as framework, architecture or pre-training contribute to the semantic understanding of instance segmentation. To answer this question, we consider a special case of robustness and compare pre-trained models on a challenging benchmark for object-centric, out-of-distribution texture. We do not introduce another method in this work. Instead, we take a step back and evaluate a broad range of existing literature. This includes Cascade and Mask R-CNN, Swin Transformer, BMask, YOLACT(++), DETR, BCNet, SOTR and SOLOv2. We find that YOLACT++, SOTR and SOLOv2 are significantly more robust to out-of-distribution texture than other frameworks. In addition, we show that deeper and dynamic architectures improve robustness whereas training schedules, data augmentation and pre-training have only a minor impact. In summary we evaluate 68 models on 61 versions of MS COCO for a total of 4148 evaluations.
@article{arxiv.2401.09109,
title = {Trapped in texture bias? A large scale comparison of deep instance segmentation},
author = {Johannes Theodoridis and Jessica Hofmann and Johannes Maucher and Andreas Schilling},
journal= {arXiv preprint arXiv:2401.09109},
year = {2024}
}
Comments
Accepted at ECCV 2022. Code: https://github.com/JohannesTheo/trapped-in-texture-bias