SI-Score: An image dataset for fine-grained analysis of robustness to object location, rotation and size
Computer Vision and Pattern Recognition
2021-04-12 v1 Artificial Intelligence
Machine Learning
Abstract
Before deploying machine learning models it is critical to assess their robustness. In the context of deep neural networks for image understanding, changing the object location, rotation and size may affect the predictions in non-trivial ways. In this work we perform a fine-grained analysis of robustness with respect to these factors of variation using SI-Score, a synthetic dataset. In particular, we investigate ResNets, Vision Transformers and CLIP, and identify interesting qualitative differences between these.
Cite
@article{arxiv.2104.04191,
title = {SI-Score: An image dataset for fine-grained analysis of robustness to object location, rotation and size},
author = {Jessica Yung and Rob Romijnders and Alexander Kolesnikov and Lucas Beyer and Josip Djolonga and Neil Houlsby and Sylvain Gelly and Mario Lucic and Xiaohua Zhai},
journal= {arXiv preprint arXiv:2104.04191},
year = {2021}
}
Comments
4 pages (10 pages including references and appendix), 10 figures. Accepted at the ICLR 2021 RobustML Workshop. arXiv admin note: text overlap with arXiv:2007.08558