Image caption generation systems are typically evaluated against reference outputs. We show that it is possible to predict output quality without generating the captions, based on the probability assigned by the neural model to the reference captions. Such pre-gen metrics are strongly correlated to standard evaluation metrics.
Cite
@article{arxiv.1810.05474,
title = {Pre-gen metrics: Predicting caption quality metrics without generating captions},
author = {Marc Tanti and Albert Gatt and Adrian Muscat},
journal= {arXiv preprint arXiv:1810.05474},
year = {2019}
}
Comments
13 pages, 6 figures This publication will appear in the Proceedings of the First Workshop on Shortcomings in Vision and Language (2018). DOI to be inserted later