IFNSO: Iteration-Free Newton-Schulz Orthogonalization
Abstract
The Newton-Schulz (NS) iteration has become a key technique for orthogonalization in optimizers such as Muon and for optimization on the Stiefel manifold. Despite its effectiveness, the conventional NS iteration incurs significant computational overhead due to repeated high-dimensional matrix multiplications. To overcome these limitations, we propose Iteration-Free Newton-Schulz Orthogonalization (IFNSO), a novel framework that consolidates the traditional iterative structure into a unified and Iteration-Free formulation. By analyzing the contribution of individual matrix powers, we streamline the process by removing insignificant terms and introducing a polynomial with learnable coefficients. These coefficients are optimized to ensure both superior computational efficiency and stable convergence. Extensive experiments demonstrate that IFNSO achieves superior performance compared to existing methods. Our code is available at: https://github.com/greekinRoma/Unified_Newton_Schulz_Orthogonalization.
Cite
@article{arxiv.2602.02500,
title = {IFNSO: Iteration-Free Newton-Schulz Orthogonalization},
author = {Chen Hu and Qianxi Zhao and Xiaochen Yuan and Hong Zhang and Ding Yuan and Yanbin Wu and Xiying Li},
journal= {arXiv preprint arXiv:2602.02500},
year = {2026}
}
Comments
The paper is under consideration at Pattern Recognition Letters