OLMoTrace：追踪语言模型输出溯源至数万亿训练标记
计算与语言
2025-07-09 v2
摘要
我们提出OLMoTrace，这是首个将语言模型输出溯源至其完整的多万亿标记训练数据的系统。OLMoTrace在语言模型输出片段与训练文本语料库中文档之间查找并显示逐字匹配。基于infini-gram（Liu等，2024）的扩展版本，我们的系统能在几秒钟内返回溯源结果。OLMoTrace有助于用户通过训练数据的视角来理解语言模型的行为。我们展示了如何利用它来探索事实核查、幻觉以及语言模型的创造力。OLMoTrace公开可用且完全开源。
关键词
引用
@article{arxiv.2504.07096,
title = {OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens},
author = {Jiacheng Liu and Taylor Blanton and Yanai Elazar and Sewon Min and YenSung Chen and Arnavi Chheda-Kothary and Huy Tran and Byron Bischoff and Eric Marsh and Michael Schmitz and Cassidy Trier and Aaron Sarnat and Jenna James and Jon Borchardt and Bailey Kuehl and Evie Cheng and Karen Farley and Sruthi Sreeram and Taira Anderson and David Albright and Carissa Schoenick and Luca Soldaini and Dirk Groeneveld and Rock Yuren Pang and Pang Wei Koh and Noah A. Smith and Sophie Lebrecht and Yejin Choi and Hannaneh Hajishirzi and Ali Farhadi and Jesse Dodge},
journal= {arXiv preprint arXiv:2504.07096},
year = {2025}
}
备注
ACL 2025 demo track