Although exact term match between queries and documents is the dominant method to perform first-stage retrieval, we propose a different approach, called RepBERT, to represent documents and queries with fixed-length contextualized embeddings. The inner products of query and document embeddings are regarded as relevance scores. On MS MARCO Passage Ranking task, RepBERT achieves state-of-the-art results among all initial retrieval techniques. And its efficiency is comparable to bag-of-words methods.
@article{arxiv.2006.15498,
title = {RepBERT: Contextualized Text Embeddings for First-Stage Retrieval},
author = {Jingtao Zhan and Jiaxin Mao and Yiqun Liu and Min Zhang and Shaoping Ma},
journal= {arXiv preprint arXiv:2006.15498},
year = {2020}
}
Comments
For corresponding code and data, see https://github.com/jingtaozhan/RepBERT-Index