Training deep neural networks at scale can benefit from domain decomposition, where the network is split into subdomains trained in parallel and coupled by a global trust-region mechanism. Building on the Additively Preconditioned Trust-Region Strategy (APTS), we propose a non-monotone variant with a nonlinear additive Schwarz preconditioner that combines parallel subdomain corrections with global coarse-space directions. A windowed acceptance criterion allows controlled objective increases, avoiding needless rejection of effective coarse steps. The resulting non-monotone APTS (NAPTS) preserves accuracy while reducing CPU time by 30\% and cutting rejected steps to one third of those in APTS.
Cite
@article{arxiv.2605.14860,
title = {A Non-Monotone Preconditioned Trust-Region Method for Neural Network Training},
author = {Andrea Angino and Bindi Çapriqi and Shega Likaj and Ken Trotti and Rolf Krause},
journal= {arXiv preprint arXiv:2605.14860},
year = {2026}
}