Training of deep residual networks with stochastic MG/OPT
Machine Learning
2021-08-10 v1
Abstract
We train deep residual networks with a stochastic variant of the nonlinear multigrid method MG/OPT. To build the multilevel hierarchy, we use the dynamical systems viewpoint specific to residual networks. We report significant speed-ups and additional robustness for training MNIST on deep residual networks. Our numerical experiments also indicate that multilevel training can be used as a pruning technique, as many of the auxiliary networks have accuracies comparable to the original network.
Keywords
Cite
@article{arxiv.2108.04052,
title = {Training of deep residual networks with stochastic MG/OPT},
author = {Cyrill von Planta and Alena Kopanicakova and Rolf Krause},
journal= {arXiv preprint arXiv:2108.04052},
year = {2021}
}