Since their inception, Vision Transformers (ViTs) have emerged as a compelling alternative to Convolutional Neural Networks (CNNs) across a wide spectrum of tasks. ViTs exhibit notable characteristics, including global attention, resilience against occlusions, and adaptability to distribution shifts. One underexplored aspect of ViTs is their potential for multi-attribute learning, referring to their ability to simultaneously grasp multiple attribute-related tasks. In this paper, we delve into the multi-attribute learning capability of ViTs, presenting a straightforward yet effective strategy for training various attributes through a single ViT network as distinct tasks. We assess the resilience of multi-attribute ViTs against adversarial attacks and compare their performance against ViTs designed for single attributes. Moreover, we further evaluate the robustness of multi-attribute ViTs against a recent transformer based attack called Patch-Fool. Our empirical findings on the CelebA dataset provide validation for our assertion. Our code is available at https://github.com/hananshafi/MTL-ViT
@article{arxiv.2402.08070,
title = {Multi-Attribute Vision Transformers are Efficient and Robust Learners},
author = {Hanan Gani and Nada Saadi and Noor Hussein and Karthik Nandakumar},
journal= {arXiv preprint arXiv:2402.08070},
year = {2024}
}
Comments
Accepted at IEEE ICIP 2024. arXiv admin note: text overlap with arXiv:2207.08677 by other authors