Does This Summary Answer My Question? Modeling Query-Focused Summary Readers with Rational Speech Acts
Abstract
Query-focused summarization (QFS) is the task of generating a summary in response to a user-written query. Despite its user-oriented nature, there has been limited work in QFS in explicitly considering a user's understanding of a generated summary, potentially causing QFS systems to underperform at inference time. In this paper, we adapt the Rational Speech Act (RSA) framework, a model of human communication, to explicitly model a reader's understanding of a query-focused summary and integrate it within the generation method of existing QFS systems. In particular, we introduce the answer reconstruction objective which approximates a reader's understanding of a summary by their ability to use it to reconstruct the answer to their initial query. Using this objective, we are able to re-rank candidate summaries generated by existing QFS systems and select summaries that better align with their corresponding query and reference summary. More generally, our study suggests that a simple and effective way of improving a language generation system designed for a user-centered task may be to explicitly incorporate its user requirements into the system's generation procedure.
Cite
@article{arxiv.2411.06524,
title = {Does This Summary Answer My Question? Modeling Query-Focused Summary Readers with Rational Speech Acts},
author = {Cesare Spinoso-Di Piano and Jackie Chi Kit Cheung},
journal= {arXiv preprint arXiv:2411.06524},
year = {2024}
}