English

Chebyshev Moment Regularization (CMR): Condition-Number Control with Moment Shaping

Machine Learning 2025-10-28 v1 Numerical Analysis Numerical Analysis

Abstract

We introduce \textbf{Chebyshev Moment Regularization (CMR)}, a simple, architecture-agnostic loss that directly optimizes layer spectra. CMR jointly controls spectral edges via a log-condition proxy and shapes the interior via Chebyshev moments, with a decoupled, capped mixing rule that preserves task gradients. We prove strictly monotone descent for the condition proxy, bounded moment gradients, and orthogonal invariance. In an adversarial ``κ\kappa-stress'' setting (MNIST, 15-layer MLP), \emph{compared to vanilla training}, CMR reduces mean layer condition numbers by  ⁣103\sim\!10^3 (from 3.9 ⁣× ⁣103\approx3.9\!\times\!10^3 to 3.4\approx3.4 in 5 epochs), increases average gradient magnitude, and restores test accuracy ( 10% ⁣ ⁣86%\approx10\%\!\to\!\approx86\% ). These results support \textbf{optimization-driven spectral preconditioning}: directly steering models toward well-conditioned regimes for stable, accurate learning.

Cite

@article{arxiv.2510.21772,
  title  = {Chebyshev Moment Regularization (CMR): Condition-Number Control with Moment Shaping},
  author = {Jinwoo Baek},
  journal= {arXiv preprint arXiv:2510.21772},
  year   = {2025}
}

Comments

15 pages

R2 v1 2026-07-01T07:04:34.020Z