Infinite-dimensional Folded-in-time Deep Neural Networks
Abstract
The method recently introduced in arXiv:2011.10115 realizes a deep neural network with just a single nonlinear element and delayed feedback. It is applicable for the description of physically implemented neural networks. In this work, we present an infinite-dimensional generalization, which allows for a more rigorous mathematical analysis and a higher flexibility in choosing the weight functions. Precisely speaking, the weights are described by Lebesgue integrable functions instead of step functions. We also provide a functional back-propagation algorithm, which enables gradient descent training of the weights. In addition, with a slight modification, our concept realizes recurrent neural networks.
Cite
@article{arxiv.2101.02966,
title = {Infinite-dimensional Folded-in-time Deep Neural Networks},
author = {Florian Stelzer and Serhiy Yanchuk},
journal= {arXiv preprint arXiv:2101.02966},
year = {2021}
}
Comments
The manuscript describes a delay system for deep learning (a continuous version of the Fit-DNN method from [1]). The main result is an algorithm for calculating functional derivatives for gradient descent training. We withdraw our paper because we discovered a more efficient existing algorithm in [2]. [1] Stelzer et al., arXiv:2011.10115. [2] Furuhata et al., Phys. Rev. Applied 15, 034092