Are skip connections necessary for biologically plausible learning rules?
Neural and Evolutionary Computing
2020-01-07 v1 Machine Learning
Machine Learning
Abstract
Backpropagation is the workhorse of deep learning, however, several other biologically-motivated learning rules have been introduced, such as random feedback alignment and difference target propagation. None of these methods have produced a competitive performance against backpropagation. In this paper, we show that biologically-motivated learning rules with skip connections between intermediate layers can perform as well as backpropagation on the MNIST dataset and are robust to various sets of hyper-parameters.
Cite
@article{arxiv.2001.01647,
title = {Are skip connections necessary for biologically plausible learning rules?},
author = {Daniel Jiwoong Im and Rutuja Patil and Kristin Branson},
journal= {arXiv preprint arXiv:2001.01647},
year = {2020}
}