Current image description generation models do not transfer well to the task of describing human faces. To encourage the development of more human-focused descriptions, we developed a new data set of facial descriptions based on the CelebA image data set. We describe the properties of this data set, and present results from a face description generator trained on it, which explores the feasibility of using transfer learning from VGGFace/ResNet CNNs. Comparisons are drawn through both automated metrics and human evaluation by 76 English-speaking participants. The descriptions generated by the VGGFace-LSTM + Attention model are closest to the ground truth according to human evaluation whilst the ResNet-LSTM + Attention model obtained the highest CIDEr and CIDEr-D results (1.252 and 0.686 respectively). Together, the new data set and these experimental results provide data and baselines for future work in this area.
@article{arxiv.2205.12342,
title = {Face2Text revisited: Improved data set and baseline results},
author = {Marc Tanti and Shaun Abdilla and Adrian Muscat and Claudia Borg and Reuben A. Farrugia and Albert Gatt},
journal= {arXiv preprint arXiv:2205.12342},
year = {2022}
}
Comments
7 pages, 5 figures, 4 tables, to appear in LREC 2022 (P-VLAM workshop)