AXLearn：模块化、硬件无关的大模型训练
机器学习
2026-02-20 v3
摘要
AXLearn 是一个生产级系统，旨在促进大型深度学习模型的可扩展高性能训练。与其他领先的深度学习系统相比，AXLearn 独特地侧重于模块化和对硬件无关训练的支持。AXLearn 的软件组件之间的内部接口遵循严格的封装，允许在不同的硬件基础设施上快速组装不同组件，以促进模型开发和实验。与领先的训练系统中线性或二次复杂度相比，AXLearn 在系统组件规模化时保持恒定的复杂度。这使得将旋转位置嵌入 (RoPE) 等功能集成到 AXLearn 中的上百个模块仅需 10 行代码，而其他系统则需要数百行。同时，AXLearn 的性能与领先的训练系统相当。最后，我们分享在 Apple 开发和运行 AXLearn 方面的经验。
引用
@article{arxiv.2507.05411,
title = {AXLearn: Modular, Hardware-Agnostic Large Model Training},
author = {Mark Lee and Chang Lan and Tom Gunter and John Peebles and Hanzhi Zhou and Kelvin Zou and Sneha Bangalore and Chung-Cheng Chiu and Nan Du and Xianzhi Du and Philipp Dufter and Ruixuan Hou and Haoshuo Huang and Dongseong Hwang and Xiang Kong and Jinhao Lei and Tao Lei and Meng Li and Li Li and Jiarui Lu and Zhiyun Lu and Yiping Ma and David Qiu and Vivek Rathod and Senyu Tong and Zhucheng Tu and Jianyu Wang and Yongqiang Wang and Zirui Wang and Floris Weers and Sam Wiseman and Guoli Yin and Bowen Zhang and Xiyou Zhou and Danyang Zhuo and Cheng Leong and Ruoming Pang},
journal= {arXiv preprint arXiv:2507.05411},
year = {2026}
}