English

From Scenes to Elements: Multi-Granularity Evidence Retrieval for Verifiable Multimodal RAG

Computation and Language 2026-05-15 v1

Abstract

Multimodal Retrieval-Augmented Generation (RAG) systems retrieve evidence at coarse granularities (entire images or scenes), creating a mismatch with fine-grained user queries and making failures unverifiable. We introduce GranuVistaVQA, a multimodal benchmark featuring real-world landmarks with element-level annotations across multiple viewpoints, capturing the partial observation challenge where individual images contain only subsets of entities. We further propose GranuRAG, a multi-granularity framework that treats visual elements as first-class retrieval units through three stages: element-level detection and classification, multi-granularity cross-modal alignment for evidence retrieval, and attribution-constrained generation. By grounding retrieval at the element level rather than relying on implicit attention, our approach enables transparent error diagnosis. Experiments demonstrate that GranuRAG achieves up to 29.2% improvement over six strong baselines for this task.

Keywords

Cite

@article{arxiv.2605.15019,
  title  = {From Scenes to Elements: Multi-Granularity Evidence Retrieval for Verifiable Multimodal RAG},
  author = {Guanhua Chen and Chuyue Huang and Yutong Yao and Shudong Liu and Xueqing Song and Lidia S. Chao and Derek F. Wong},
  journal= {arXiv preprint arXiv:2605.15019},
  year   = {2026}
}