Relation extraction (RE) aims to identify semantic relations between entities in unstructured text. Although recent work extends traditional RE to multimodal scenarios, most approaches still adopt classification-based paradigms with fused multimodal features, representing relations as discrete labels. This paradigm has two significant limitations: (1) it overlooks structural constraints like entity types and positional cues, and (2) it lacks semantic expressiveness for fine-grained relation understanding. We propose \underline{R}etrieval \underline{O}ver \underline{C}lassification (ROC), a novel framework that reformulates multimodal RE as a retrieval task driven by relation semantics. ROC integrates entity type and positional information through a multimodal encoder, expands relation labels into natural language descriptions using a large language model, and aligns entity-relation pairs via semantic similarity-based contrastive learning. Experiments show that our method achieves state-of-the-art performance on the benchmark datasets MNRE and MORE and exhibits stronger robustness and interpretability.
@article{arxiv.2509.21151,
title = {Retrieval over Classification: Integrating Relation Semantics for Multimodal Relation Extraction},
author = {Lei Hei and Tingjing Liao and Yingxin Pei and Yiyang Qi and Jiaqi Wang and Ruiting Li and Feiliang Ren},
journal= {arXiv preprint arXiv:2509.21151},
year = {2025}
}