Changing speaker names consistently throughout a dialogue should not affect its meaning and corresponding outputs for text generation from dialogues. However, pre-trained language models, serving as the backbone for dialogue-processing tasks, have shown to be sensitive to nuances. This may result in unfairness in real-world applications. No comprehensive analysis of this problem has been done in the past. In this work, we propose to quantitatively measure a model's sensitivity on speaker names, and comprehensively evaluate a number of known methods for reducing speaker name sensitivity, including a novel approach of our own. Extensive experiments on multiple datasets provide a benchmark for this problem and show the favorable performance of our approach in sensitivity reduction and quality of generation.
@article{arxiv.2305.13833,
title = {Reducing Sensitivity on Speaker Names for Text Generation from Dialogues},
author = {Qi Jia and Haifeng Tang and Kenny Q. Zhu},
journal= {arXiv preprint arXiv:2305.13833},
year = {2023}
}