The challenge of Out-Of-Distribution (OOD) robustness remains a critical hurdle towards deploying deep vision models. Vision-Language Models (VLMs) have recently achieved groundbreaking results. VLM-based open-vocabulary object detection extends the capabilities of traditional object detection frameworks, enabling the recognition and classification of objects beyond predefined categories. Investigating OOD robustness in recent open-vocabulary object detection is essential to increase the trustworthiness of these models. This study presents a comprehensive robustness evaluation of the zero-shot capabilities of three recent open-vocabulary (OV) foundation object detection models: OWL-ViT, YOLO World, and Grounding DINO. Experiments carried out on the robustness benchmarks COCO-O, COCO-DC, and COCO-C encompassing distribution shifts due to information loss, corruption, adversarial attacks, and geometrical deformation, highlighting the challenges of the model's robustness to foster the research for achieving robustness. Project page: https://prakashchhipa.github.io/projects/ovod_robustness
@article{arxiv.2405.14874,
title = {Open-Vocabulary Object Detectors: Robustness Challenges under Distribution Shifts},
author = {Prakash Chandra Chhipa and Kanjar De and Meenakshi Subhash Chippa and Rajkumar Saini and Marcus Liwicki},
journal= {arXiv preprint arXiv:2405.14874},
year = {2024}
}
Comments
Accepted at 2024 European Conference on Computer Vision Workshops (ECCVW). Project page - https://prakashchhipa.github.io/projects/ovod_robustness