Neural networks for computer vision extract uninterpretable features despite achieving high accuracy on benchmarks. In contrast, humans can explain their predictions using succinct and intuitive descriptions. To incorporate explainability into neural networks, we train a vision model whose feature representations are text. We show that such a model can effectively classify ImageNet images, and we discuss the challenges we encountered when training it.
@article{arxiv.2307.00028,
title = {Seeing in Words: Learning to Classify through Language Bottlenecks},
author = {Khalid Saifullah and Yuxin Wen and Jonas Geiping and Micah Goldblum and Tom Goldstein},
journal= {arXiv preprint arXiv:2307.00028},
year = {2023}
}
Comments
5 pages, 2 figures, Published as a Tiny Paper at ICLR 2023