FACTS Leaderboard：用于大型语言模型事实性评估的综合基准
计算与语言
2025-12-12 v1 人工智能
摘要
我们介绍FACTS Leaderboard，这是一个在线评分榜套件及其相关的基准测试集合，全面评估语言模型在不同情景下生成事实准确文本的能力。该套件通过聚合模型在四个不同子评分榜上的表现，提供整体的事实性衡量：（1）FACTS多模态版，衡量对图像问题的回答的事实性；（2）FACTS参数版，评估模型通过内部参数回答闭合书籍事实问题的世界知识；（3）FACTS搜索版，评估信息寻求情境下的事实性，其中模型必须使用搜索API；（4）FACTS grounding（v2）版，评估长篇回答是否基于提供的文件， featuring显著改进的评判模型。每个子评分榜都采用自动化评判模型对模型回答进行评分，最终套件得分是四个组成部分的平均值，旨在提供对模型整体事实性的稳健且平衡的评估。FACTS Leaderboard套件将持续维护，包含公共和私人拆分，以允许外部参与同时维护其完整性。可在 https://www.kaggle.com/benchmarks/google/facts 访问。
关键词
引用
@article{arxiv.2512.10791,
title = {The FACTS Leaderboard: A Comprehensive Benchmark for Large Language Model Factuality},
author = {Aileen Cheng and Alon Jacovi and Amir Globerson and Ben Golan and Charles Kwong and Chris Alberti and Connie Tao and Eyal Ben-David and Gaurav Singh Tomar and Lukas Haas and Yonatan Bitton and Adam Bloniarz and Aijun Bai and Andrew Wang and Anfal Siddiqui and Arturo Bajuelos Castillo and Aviel Atias and Chang Liu and Corey Fry and Daniel Balle and Deepanway Ghosal and Doron Kukliansky and Dror Marcus and Elena Gribovskaya and Eran Ofek and Honglei Zhuang and Itay Laish and Jan Ackermann and Lily Wang and Meg Risdal and Megan Barnes and Michael Fink and Mohamed Amin and Moran Ambar and Natan Potikha and Nikita Gupta and Nitzan Katz and Noam Velan and Ofir Roval and Ori Ram and Polina Zablotskaia and Prathamesh Bang and Priyanka Agrawal and Rakesh Ghiya and Sanjay Ganapathy and Simon Baumgartner and Sofia Erell and Sushant Prakash and Thibault Sellam and Vikram Rao and Xuanhui Wang and Yaroslav Akulov and Yulong Yang and Zhen Yang and Zhixin Lai and Zhongru Wu and Anca Dragan and Avinatan Hassidim and Fernando Pereira and Slav Petrov and Srinivasan Venkatachary and Tulsee Doshi and Yossi Matias and Sasha Goldshtein and Dipanjan Das},
journal= {arXiv preprint arXiv:2512.10791},
year = {2025}
}