Deep Learning: Generalization Requires Deep Compositional Feature Space Design

Mrinal Haloi

Deep Learning: Generalization Requires Deep Compositional Feature Space Design

Machine Learning 2017-07-11 v2 Machine Learning

Authors: Mrinal Haloi

Abstract

Generalization error defines the discriminability and the representation power of a deep model. In this work, we claim that feature space design using deep compositional function plays a significant role in generalization along with explicit and implicit regularizations. Our claims are being established with several image classification experiments. We show that the information loss due to convolution and max pooling can be marginalized with the compositional design, improving generalization performance. Also, we will show that learning rate decay acts as an implicit regularizer in deep model training.

Keywords

generalization in machine learning representation learning regularization

Cite

@article{arxiv.1706.01983,
  title  = {Deep Learning: Generalization Requires Deep Compositional Feature Space Design},
  author = {Mrinal Haloi},
  journal= {arXiv preprint arXiv:1706.01983},
  year   = {2017}
}

Comments

fig added, with minor typo corrections

Related papers

View all related →