Learning to Reject with a Fixed Predictor: Application to Decontextualization
Machine Learning
2023-02-01 v2
Abstract
We study the problem of classification with a reject option for a fixed predictor, applicable in natural language processing. We introduce a new problem formulation for this scenario, and an algorithm minimizing a new surrogate loss function. We provide a complete theoretical analysis of the surrogate loss function with a strong -consistency guarantee. For evaluation, we choose the decontextualization task, and provide a manually-labelled dataset of examples. Our algorithm significantly outperforms the baselines considered, with a improvement in coverage when halving the error rate, which is only away from the theoretical limit.
Cite
@article{arxiv.2301.09044,
title = {Learning to Reject with a Fixed Predictor: Application to Decontextualization},
author = {Christopher Mohri and Daniel Andor and Eunsol Choi and Michael Collins},
journal= {arXiv preprint arXiv:2301.09044},
year = {2023}
}