Stability of the Decoupled Extended Kalman Filter Learning Algorithm in LSTM-Based Online Learning
Abstract
We investigate the convergence and stability properties of the decoupled extended Kalman filter learning algorithm (DEKF) within the long-short term memory network (LSTM) based online learning framework. For this purpose, we model DEKF as a perturbed extended Kalman filter and derive sufficient conditions for its stability during LSTM training. We show that if the perturbations -- introduced due to decoupling -- stay bounded, DEKF learns LSTM parameters with similar convergence and stability properties of the global extended Kalman filter learning algorithm. We verify our results with several numerical simulations and compare DEKF with other LSTM training methods. In our simulations, we also observe that the well-known hyper-parameter selection approaches used for DEKF in the literature satisfy our conditions.
Keywords
Cite
@article{arxiv.1911.12258,
title = {Stability of the Decoupled Extended Kalman Filter Learning Algorithm in LSTM-Based Online Learning},
author = {Nuri Mert Vural and Fatih Ilhan and Suleyman S. Kozat},
journal= {arXiv preprint arXiv:1911.12258},
year = {2021}
}
Comments
This paper was an early draft of the presented results. We have written and published another paper (arXiv:1911.12258) where we have improved on the material in this paper. The published paper covers most of the material presented in this paper as well. Therefore, we remove this paper from Arxiv and refer the interested readers to arXiv:1911.12258