Over-Sampling in a Deep Neural Network
Machine Learning
2015-02-13 v1 Neural and Evolutionary Computing
Abstract
Deep neural networks (DNN) are the state of the art on many engineering problems such as computer vision and audition. A key factor in the success of the DNN is scalability - bigger networks work better. However, the reason for this scalability is not yet well understood. Here, we interpret the DNN as a discrete system, of linear filters followed by nonlinear activations, that is subject to the laws of sampling theory. In this context, we demonstrate that over-sampled networks are more selective, learn faster and learn more robustly. Our findings may ultimately generalize to the human brain.
Cite
@article{arxiv.1502.03648,
title = {Over-Sampling in a Deep Neural Network},
author = {Andrew J. R. Simpson},
journal= {arXiv preprint arXiv:1502.03648},
year = {2015}
}