通过检索-转换表头桥接潜在推理与目标语言生成
计算与语言
2026-03-02 v2
摘要
最近的研究识别了Transformer中一部分注意力头作为检索头，负责从上下文中检索信息。在本工作中，我们首先在多语言语境中探讨检索头。在多语言语言模型中，我们发现检索头常常在多个语言之间共享。将研究扩展到跨语言场景，我们识别了检索-转换头（RTH），其管控转换到特定目标语言输出的过程。我们的实验表明，RTH与检索头不同，对多语言LLM的链式思考推理更为关键。在四个多语言基准（MMLU-ProX、MGSM、MLQA和XQuaD）和两个模型家族（Qwen-2.5和Llama-3.1）上，我们展示了屏蔽RTH导致的性能下降幅度大于屏蔽检索头（RH）。我们的工作通过隔离负责映射到目标语言的注意力头，推进了对多语言LM的理解。
引用
@article{arxiv.2602.22453,
title = {Bridging Latent Reasoning and Target-Language Generation via Retrieval-Transition Heads},
author = {Shaswat Patel and Vishvesh Trivedi and Yue Han and Yihuai Hong and Eunsol Choi},
journal= {arXiv preprint arXiv:2602.22453},
year = {2026}
}
备注
In the paper, there are still many statements that are unclear and lack sufficient justification. Since it is difficult for us to estimate how much time would be required to properly revise and correct these issues, we would like to request a withdrawal of the paper in this moment. Thank you!