In classic instruction following, language like "I'd like the JetBlue flight" maps to actions (e.g., selecting that flight). However, language also conveys information about a user's underlying reward function (e.g., a general preference for JetBlue), which can allow a model to carry out desirable actions in new contexts. We present a model that infers rewards from language pragmatically: reasoning about how speakers choose utterances not only to elicit desired actions, but also to reveal information about their preferences. On a new interactive flight-booking task with natural language, our model more accurately infers rewards and predicts optimal actions in unseen environments, in comparison to past work that first maps language to actions (instruction following) and then maps actions to rewards (inverse reinforcement learning).
@article{arxiv.2204.02515,
title = {Inferring Rewards from Language in Context},
author = {Jessy Lin and Daniel Fried and Dan Klein and Anca Dragan},
journal= {arXiv preprint arXiv:2204.02515},
year = {2022}
}
Comments
ACL 2022. Code and dataset: https://github.com/jlin816/rewards-from-language