Particle Filter Recurrent Neural Networks
Abstract
Recurrent neural networks (RNNs) have been extraordinarily successful for prediction with sequential data. To tackle highly variable and noisy real-world data, we introduce Particle Filter Recurrent Neural Networks (PF-RNNs), a new RNN family that explicitly models uncertainty in its internal structure: while an RNN relies on a long, deterministic latent state vector, a PF-RNN maintains a latent state distribution, approximated as a set of particles. For effective learning, we provide a fully differentiable particle filter algorithm that updates the PF-RNN latent state distribution according to the Bayes rule. Experiments demonstrate that the proposed PF-RNNs outperform the corresponding standard gated RNNs on a synthetic robot localization dataset and 10 real-world sequence prediction datasets for text classification, stock price prediction, etc.
Cite
@article{arxiv.1905.12885,
title = {Particle Filter Recurrent Neural Networks},
author = {Xiao Ma and Peter Karkus and David Hsu and Wee Sun Lee},
journal= {arXiv preprint arXiv:1905.12885},
year = {2019}
}
Comments
Accepted to AAAI 2020