Framing RNN as a kernel method: A neural ODE approach
Machine Learning
2021-11-01 v2 Machine Learning
Abstract
Building on the interpretation of a recurrent neural network (RNN) as a continuous-time neural differential equation, we show, under appropriate conditions, that the solution of a RNN can be viewed as a linear function of a specific feature set of the input sequence, known as the signature. This connection allows us to frame a RNN as a kernel method in a suitable reproducing kernel Hilbert space. As a consequence, we obtain theoretical guarantees on generalization and stability for a large class of recurrent networks. Our results are illustrated on simulated datasets.
Keywords
Cite
@article{arxiv.2106.01202,
title = {Framing RNN as a kernel method: A neural ODE approach},
author = {Adeline Fermanian and Pierre Marion and Jean-Philippe Vert and Gérard Biau},
journal= {arXiv preprint arXiv:2106.01202},
year = {2021}
}
Comments
33 pages, 7 figures, accepted for an oral presentation at NeurIPS 2021