W2T：LoRA 权重已知它们能做什么
机器学习
2026-03-18 v1
摘要
每个 LoRA 检查点都以低秩权重矩阵的形式紧凑存储任务特定的更新，为将大型语言模型适应新任务和领域提供了高效方法。从原理上说，这些权重已经编码了适配器的行为及其性能。本文我们询问，这些信息能否直接从权重中读取，而无需运行基础模型或访问训练数据。一个关键障碍是，单个 LoRA 更新可以无限多种方式进行因式分解。若不解决这种歧义，训练的模型可能会拟合特定的因式分解，而非底层更新。为此，我们提出了 \methodfull，将每个 LoRA 更新通过 QR 分解随后进行 SVD，映射到一个可证明的规范形式，以便所有等效因式分解共享相同的表示。 resulting components are then tokenized and processed by a Transformer to produce a weight-space embedding. Across language and vision LoRA collections, W2T achieves strong results on attribute classification, performance prediction, and adapter retrieval, demonstrating that LoRA weights reliably indicate model behavior once factorization ambiguity is removed. Code is available at https://github.com/xiaolonghan2000/Weight2Token.
关键词
引用
@article{arxiv.2603.15990,
title = {W2T: LoRA Weights Already Know What They Can Do},
author = {Xiaolong Han and Ferrante Neri and Zijian Jiang and Fang Wu and Yanfang Ye and Lu Yin and Zehong Wang},
journal= {arXiv preprint arXiv:2603.15990},
year = {2026}
}