English

CCCaption: Dual-Reward Reinforcement Learning for Complete and Correct Image Captioning

Computer Vision and Pattern Recognition 2026-03-31 v2 Artificial Intelligence

Abstract

Image captioning remains a fundamental task for vision language understanding, yet ground-truth supervision still relies predominantly on human-annotated references. Because human annotations reflect subjective preferences and expertise, ground-truth captions are often incomplete or even incorrect, which in turn limits caption models. We argue that caption quality should be assessed by two objective aspects: completeness (does the caption cover all salient visual facts?) and correctness (are the descriptions true with respect to the image?). To this end, we introduce CCCaption: a dual-reward reinforcement learning framework with a dedicated fine-tuning corpus that explicitly optimizes these properties to generate \textbf{C}omplete and \textbf{C}orrect \textbf{Captions}. For completeness, we use diverse LVLMs to disentangle the image into a set of visual queries, and reward captions that answer more of these queries, with a dynamic query sampling strategy to improve training efficiency. For correctness, we penalize captions that contain hallucinations by validating the authenticity of sub-caption queries, which are derived from the caption decomposition. Our symmetric dual-reward optimization jointly maximizes completeness and correctness, guiding models toward captions that better satisfy these objective criteria. Extensive experiments across standard captioning benchmarks show consistent improvements, offering a principled path to training caption models beyond human-annotation imitation.

Keywords

Cite

@article{arxiv.2602.21655,
  title  = {CCCaption: Dual-Reward Reinforcement Learning for Complete and Correct Image Captioning},
  author = {Zhijiang Tang and Linhua Wang and Jiaxin Qi and Weihao Jiang and Peng Hou and Anxiang Zeng and Jianqiang Huang},
  journal= {arXiv preprint arXiv:2602.21655},
  year   = {2026}
}

Comments

Accept by CVPR 2026

R2 v1 2026-07-01T10:51:28.427Z