Ky 范数及更远：矩阵优化中的双范数与组合

本文探讨了 various matrix norms for optimizing functions of weight matrices, a crucial problem in training large language models. 通过拓展 Muon 更新所基于的谱范数，我们利用 Ky Fan $k$-范数的对偶形式，引入一族称为 Fanions 的类似 Muon 的算法，这些算法与 Dion closely related. 通过 working with convex combinations of the Ky Fan $k$-norms with either the Frobenius norm or the $l_\infty$ norm, we construct the families of F-Fanions and S-Fanions, respectively. Their most prominent members are F-Muon and S-Muon. We complement our theoretical analysis with an extensive empirical study of these algorithms across a wide range of tasks and settings, demonstrating that F-Muon and S-Muon consistently match Muon's performance, while outperforming vanilla Muon on a synthetic linear least squares problem.