Chebyshev Moment Regularization (CMR): Condition-Number Control with Moment Shaping
Abstract
We introduce \textbf{Chebyshev Moment Regularization (CMR)}, a simple, architecture-agnostic loss that directly optimizes layer spectra. CMR jointly controls spectral edges via a log-condition proxy and shapes the interior via Chebyshev moments, with a decoupled, capped mixing rule that preserves task gradients. We prove strictly monotone descent for the condition proxy, bounded moment gradients, and orthogonal invariance. In an adversarial ``-stress'' setting (MNIST, 15-layer MLP), \emph{compared to vanilla training}, CMR reduces mean layer condition numbers by (from to in 5 epochs), increases average gradient magnitude, and restores test accuracy ( ). These results support \textbf{optimization-driven spectral preconditioning}: directly steering models toward well-conditioned regimes for stable, accurate learning.
Cite
@article{arxiv.2510.21772,
title = {Chebyshev Moment Regularization (CMR): Condition-Number Control with Moment Shaping},
author = {Jinwoo Baek},
journal= {arXiv preprint arXiv:2510.21772},
year = {2025}
}
Comments
15 pages