Deep networks, composed of multiple layers of hierarchical distributed representations, tend to learn low-level features in initial layers and transition to high-level features towards final layers. Paradigms such as transfer learning, multi-task learning, and continual learning leverage this notion of generic hierarchical distributed representations to share knowledge across datasets and tasks. Herein, we study the layer-wise transferability of representations in deep networks across a few datasets and tasks and note some interesting empirical observations.
@article{arxiv.1811.12273,
title = {On the Transferability of Representations in Neural Networks Between Datasets and Tasks},
author = {Haytham M. Fayek and Lawrence Cavedon and Hong Ren Wu},
journal= {arXiv preprint arXiv:1811.12273},
year = {2018}
}
Comments
Accepted Paper in the Continual Learning Workshop, NeurIPS 2018