Visual question answering (VQA) methods in remote sensing (RS) aim to answer natural language questions with respect to an RS image. Most of the existing methods require a large amount of computational resources, which limits their application in operational scenarios in RS. To address this issue, in this paper we present an effective lightweight transformer-based VQA in RS (LiT-4-RSVQA) architecture for efficient and accurate VQA in RS. Our architecture consists of: i) a lightweight text encoder module; ii) a lightweight image encoder module; iii) a fusion module; and iv) a classification module. The experimental results obtained on a VQA benchmark dataset demonstrate that our proposed LiT-4-RSVQA architecture provides accurate VQA results while significantly reducing the computational requirements on the executing hardware. Our code is publicly available at https://git.tu-berlin.de/rsim/lit4rsvqa.
@article{arxiv.2306.00758,
title = {LiT-4-RSVQA: Lightweight Transformer-based Visual Question Answering in Remote Sensing},
author = {Leonard Hackel and Kai Norman Clasen and Mahdyar Ravanbakhsh and Begüm Demir},
journal= {arXiv preprint arXiv:2306.00758},
year = {2023}
}
Comments
Accepted at IEEE International Geoscience and Remote Sensing Symposium 2023