English

ReCap: Event-Aware Image Captioning with Article Retrieval and Semantic Gaussian Normalization

Computer Vision and Pattern Recognition 2025-09-03 v1

Abstract

Image captioning systems often produce generic descriptions that fail to capture event-level semantics which are crucial for applications like news reporting and digital archiving. We present ReCap, a novel pipeline for event-enriched image retrieval and captioning that incorporates broader contextual information from relevant articles to generate narrative-rich, factually grounded captions. Our approach addresses the limitations of standard vision-language models that typically focus on visible content while missing temporal, social, and historical contexts. ReCap comprises three integrated components: (1) a robust two-stage article retrieval system using DINOv2 embeddings with global feature similarity for initial candidate selection followed by patch-level mutual nearest neighbor similarity re-ranking; (2) a context extraction framework that synthesizes information from article summaries, generic captions, and original source metadata; and (3) a large language model-based caption generation system with Semantic Gaussian Normalization to enhance fluency and relevance. Evaluated on the OpenEvents V1 dataset as part of Track 1 in the EVENTA 2025 Grand Challenge, ReCap achieved a strong overall score of 0.54666, ranking 2nd on the private test set. These results highlight ReCap's effectiveness in bridging visual perception with real-world knowledge, offering a practical solution for context-aware image understanding in high-stakes domains. The code is available at https://github.com/Noridom1/EVENTA2025-Event-Enriched-Image-Captioning.

Keywords

Cite

@article{arxiv.2509.01259,
  title  = {ReCap: Event-Aware Image Captioning with Article Retrieval and Semantic Gaussian Normalization},
  author = {Thinh-Phuc Nguyen and Thanh-Hai Nguyen and Gia-Huy Dinh and Lam-Huy Nguyen and Minh-Triet Tran and Trung-Nghia Le},
  journal= {arXiv preprint arXiv:2509.01259},
  year   = {2025}
}

Comments

ACM Multimedia 2025

R2 v1 2026-07-01T05:14:56.112Z