We propose Composition Sampling, a simple but effective method to generate diverse outputs for conditional generation of higher quality compared to previous stochastic decoding strategies. It builds on recently proposed plan-based neural generation models (Narayan et al, 2021) that are trained to first create a composition of the output and then generate by conditioning on it and the input. Our approach avoids text degeneration by first sampling a composition in the form of an entity chain and then using beam search to generate the best possible text grounded to this entity chain. Experiments on summarization (CNN/DailyMail and XSum) and question generation (SQuAD), using existing and newly proposed automatic metrics together with human-based evaluation, demonstrate that Composition Sampling is currently the best available decoding strategy for generating diverse meaningful outputs.
@article{arxiv.2203.15108,
title = {A Well-Composed Text is Half Done! Composition Sampling for Diverse Conditional Generation},
author = {Shashi Narayan and Gonçalo Simões and Yao Zhao and Joshua Maynez and Dipanjan Das and Michael Collins and Mirella Lapata},
journal= {arXiv preprint arXiv:2203.15108},
year = {2022}
}