A Flow Model of Neural Networks
Machine Learning
2017-12-12 v2 Artificial Intelligence
Neural and Evolutionary Computing
Abstract
Based on a natural connection between ResNet and transport equation or its characteristic equation, we propose a continuous flow model for both ResNet and plain net. Through this continuous model, a ResNet can be explicitly constructed as a refinement of a plain net. The flow model provides an alternative perspective to understand phenomena in deep neural networks, such as why it is necessary and sufficient to use 2-layer blocks in ResNets, why deeper is better, and why ResNets are even deeper, and so on. It also opens a gate to bring in more tools from the huge area of differential equations.
Keywords
Cite
@article{arxiv.1708.06257,
title = {A Flow Model of Neural Networks},
author = {Zhen Li and Zuoqiang Shi},
journal= {arXiv preprint arXiv:1708.06257},
year = {2017}
}