Pre-trained and fine-tuned transformer models like BERT and T5 have improved the state of the art in ad-hoc retrieval and question-answering, but not as yet in high-recall information retrieval, where the objective is to retrieve substantially all relevant documents. We investigate whether the use of transformer-based models for reranking and/or featurization can improve the Baseline Model Implementation of the TREC Total Recall Track, which represents the current state of the art for high-recall information retrieval. We also introduce CALBERT, a model that can be used to continuously fine-tune a BERT-based model based on relevance feedback.
@article{arxiv.2208.06955,
title = {Continuous Active Learning Using Pretrained Transformers},
author = {Nima Sadri and Gordon V. Cormack},
journal= {arXiv preprint arXiv:2208.06955},
year = {2022}
}