Symmetry & Critical Points
Machine Learning
2024-08-27 v1 Numerical Analysis
Numerical Analysis
Optimization and Control
Machine Learning
Abstract
Critical points of an invariant function may or may not be symmetric. We prove, however, that if a symmetric critical point exists, those adjacent to it are generically symmetry breaking. This mathematical mechanism is shown to carry important implications for our ability to efficiently minimize invariant nonconvex functions, in particular those associated with neural networks.
Cite
@article{arxiv.2408.14445,
title = {Symmetry & Critical Points},
author = {Yossi Arjevani},
journal= {arXiv preprint arXiv:2408.14445},
year = {2024}
}