Enhancing BERT-Based Visual Question Answering through Keyword-Driven Sentence Selection
Abstract
The Document-based Visual Question Answering competition addresses the automatic detection of parent-child relationships between elements in multi-page documents. The goal is to identify the document elements that answer a specific question posed in natural language. This paper describes the PoliTo's approach to addressing this task, in particular, our best solution explores a text-only approach, leveraging an ad hoc sampling strategy. Specifically, our approach leverages the Masked Language Modeling technique to fine-tune a BERT model, focusing on sentences containing sensitive keywords that also occur in the questions, such as references to tables or images. Thanks to the effectiveness of this approach, we are able to achieve high performance compared to baselines, demonstrating how our solution contributes positively to this task.
Keywords
Cite
@article{arxiv.2310.09432,
title = {Enhancing BERT-Based Visual Question Answering through Keyword-Driven Sentence Selection},
author = {Davide Napolitano and Lorenzo Vaiani and Luca Cagliero},
journal= {arXiv preprint arXiv:2310.09432},
year = {2023}
}
Comments
This paper is the technical research paper of CIKM 2023 DocIU challenges. The authors received the CIKM 2023 DocIU Winner Award, sponsored by Google, Microsoft, and the Centre for data-driven geoscience