Understanding over-parameterized deep networks by geometrization
Machine Learning
2019-02-12 v1
Abstract
A complete understanding of the widely used over-parameterized deep networks is a key step for AI. In this work we try to give a geometric picture of over-parameterized deep networks using our geometrization scheme. We show that the Riemannian geometry of network complexity plays a key role in understanding the basic properties of over-parameterizaed deep networks, including the generalization, convergence and parameter sensitivity. We also point out deep networks share lots of similarities with quantum computation systems. This can be regarded as a strong support of our proposal that geometrization is not only the bible for physics, it is also the key idea to understand deep learning systems.
Cite
@article{arxiv.1902.03793,
title = {Understanding over-parameterized deep networks by geometrization},
author = {Xiao Dong and Ling Zhou},
journal= {arXiv preprint arXiv:1902.03793},
year = {2019}
}
Comments
6 pages