A main goal of Argument Mining (AM) is to analyze an author's stance. Unlike previous AM datasets focusing only on text, the shared task at the 10th Workshop on Argument Mining introduces a dataset including both text and images. Importantly, these images contain both visual elements and optical characters. Our new framework, TILFA (A Unified Framework for Text, Image, and Layout Fusion in Argument Mining), is designed to handle this mixed data. It excels at not only understanding text but also detecting optical characters and recognizing layout details in images. Our model significantly outperforms existing baselines, earning our team, KnowComp, the 1st place in the leaderboard of Argumentative Stance Classification subtask in this shared task.
@article{arxiv.2310.05210,
title = {TILFA: A Unified Framework for Text, Image, and Layout Fusion in Argument Mining},
author = {Qing Zong and Zhaowei Wang and Baixuan Xu and Tianshi Zheng and Haochen Shi and Weiqi Wang and Yangqiu Song and Ginny Y. Wong and Simon See},
journal= {arXiv preprint arXiv:2310.05210},
year = {2023}
}
Comments
Accepted to the 10th Workshop on Argument Mining, co-located with EMNLP 2023