Mixing Complexity and its Applications to Neural Networks
Machine Learning
2017-03-03 v1
Abstract
We suggest analyzing neural networks through the prism of space constraints. We observe that most training algorithms applied in practice use bounded memory, which enables us to use a new notion introduced in the study of space-time tradeoffs that we call mixing complexity. This notion was devised in order to measure the (in)ability to learn using a bounded-memory algorithm. In this paper we describe how we use mixing complexity to obtain new results on what can and cannot be learned using neural networks.
Cite
@article{arxiv.1703.00729,
title = {Mixing Complexity and its Applications to Neural Networks},
author = {Michal Moshkovitz and Naftali Tishby},
journal= {arXiv preprint arXiv:1703.00729},
year = {2017}
}