A Theory of Feature Learning
Machine Learning
2015-04-02 v1 Machine Learning
Abstract
Feature Learning aims to extract relevant information contained in data sets in an automated fashion. It is driving force behind the current deep learning trend, a set of methods that have had widespread empirical success. What is lacking is a theoretical understanding of different feature learning schemes. This work provides a theoretical framework for feature learning and then characterizes when features can be learnt in an unsupervised fashion. We also provide means to judge the quality of features via rate-distortion theory and its generalizations.
Cite
@article{arxiv.1504.00083,
title = {A Theory of Feature Learning},
author = {Brendan van Rooyen and Robert C. Williamson},
journal= {arXiv preprint arXiv:1504.00083},
year = {2015}
}