Interpolation Technique to Speed Up Gradients Propagation in Neural ODEs
Neural and Evolutionary Computing
2020-11-03 v2 Numerical Analysis
Numerical Analysis
Machine Learning
Abstract
We propose a simple interpolation-based method for the efficient approximation of gradients in neural ODE models. We compare it with the reverse dynamic method (known in the literature as "adjoint method") to train neural ODEs on classification, density estimation, and inference approximation tasks. We also propose a theoretical justification of our approach using logarithmic norm formalism. As a result, our method allows faster model training than the reverse dynamic method that was confirmed and validated by extensive numerical experiments for several standard benchmarks.
Keywords
Cite
@article{arxiv.2003.05271,
title = {Interpolation Technique to Speed Up Gradients Propagation in Neural ODEs},
author = {Talgat Daulbaev and Alexandr Katrutsa and Larisa Markeeva and Julia Gusak and Andrzej Cichocki and Ivan Oseledets},
journal= {arXiv preprint arXiv:2003.05271},
year = {2020}
}