NeurIPS 2020 EfficientQA 竞赛:系统、分析与经验教训
计算与语言
2021-09-21 v2 人工智能
摘要
我们回顾了 NeurIPS 2020 的 EfficientQA 竞赛。该竞赛聚焦于开放域问答(QA),系统以自然语言问题为输入并返回自然语言答案。竞赛旨在构建能够预测正确答案,同时满足严格磁盘内存预算的系统。这些内存预算旨在鼓励参赛者探索存储检索语料库与存储学习模型参数之间的权衡。在本报告中,我们描述了竞赛的动机与组织方式,回顾了最佳提交方案,并分析了系统预测以启发关于开放域 QA 评估的讨论。
引用
@article{arxiv.2101.00133,
title = {NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned},
author = {Sewon Min and Jordan Boyd-Graber and Chris Alberti and Danqi Chen and Eunsol Choi and Michael Collins and Kelvin Guu and Hannaneh Hajishirzi and Kenton Lee and Jennimaria Palomaki and Colin Raffel and Adam Roberts and Tom Kwiatkowski and Patrick Lewis and Yuxiang Wu and Heinrich Küttler and Linqing Liu and Pasquale Minervini and Pontus Stenetorp and Sebastian Riedel and Sohee Yang and Minjoon Seo and Gautier Izacard and Fabio Petroni and Lucas Hosseini and Nicola De Cao and Edouard Grave and Ikuya Yamada and Sonse Shimaoka and Masatoshi Suzuki and Shumpei Miyawaki and Shun Sato and Ryo Takahashi and Jun Suzuki and Martin Fajcik and Martin Docekal and Karel Ondrej and Pavel Smrz and Hao Cheng and Yelong Shen and Xiaodong Liu and Pengcheng He and Weizhu Chen and Jianfeng Gao and Barlas Oguz and Xilun Chen and Vladimir Karpukhin and Stan Peshterliev and Dmytro Okhonko and Michael Schlichtkrull and Sonal Gupta and Yashar Mehdad and Wen-tau Yih},
journal= {arXiv preprint arXiv:2101.00133},
year = {2021}
}
备注
26 pages; Published in Proceedings of Machine Learning Research (PMLR), NeurIPS 2020 Competition and Demonstration Track