Large volumes of interaction logs can be collected from NLP systems that are deployed in the real world. How can this wealth of information be leveraged? Using such interaction logs in an offline reinforcement learning (RL) setting is a promising approach. However, due to the nature of NLP tasks and the constraints of production systems, a series of challenges arise. We present a concise overview of these challenges and discuss possible solutions.
@article{arxiv.2011.02511,
title = {Offline Reinforcement Learning from Human Feedback in Real-World Sequence-to-Sequence Tasks},
author = {Julia Kreutzer and Stefan Riezler and Carolin Lawrence},
journal= {arXiv preprint arXiv:2011.02511},
year = {2021}
}
Comments
5th Workshop on Structured Prediction for NLP at ACL 2021 Previously named "Learning from Human Feedback: Challenges for Real-World Reinforcement Learning in NLP" and presented at Challenges of Real-World RL Workshop at NeurIPS 2020