We introduce pytrec_eval, a Python interface to the tree_eval information retrieval evaluation toolkit. pytrec_eval exposes the reference implementations of trec_eval within Python as a native extension. We show that pytrec_eval is around one order of magnitude faster than invoking trec_eval as a sub process from within Python. Compared to a native Python implementation of NDCG, pytrec_eval is twice as fast for practically-sized rankings. Finally, we demonstrate its effectiveness in an application where pytrec_eval is combined with Pyndri and the OpenAI Gym where query expansion is learned using Q-learning.
@article{arxiv.1805.01597,
title = {Pytrec_eval: An Extremely Fast Python Interface to trec_eval},
author = {Christophe Van Gysel and Maarten de Rijke},
journal= {arXiv preprint arXiv:1805.01597},
year = {2018}
}
Comments
SIGIR '18. The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval