Sparse Gaussian Process Temporal Difference Learning for Marine Robot Navigation
Abstract
We present a method for Temporal Difference (TD) learning that addresses several challenges faced by robots learning to navigate in a marine environment. For improved data efficiency, our method reduces TD updates to Gaussian Process regression. To make predictions amenable to online settings, we introduce a sparse approximation with improved quality over current rejection-based sparse methods. We derive the predictive value function posterior and use the moments to obtain a new algorithm for model-free policy evaluation, SPGP-SARSA. With simple changes, we show SPGP-SARSA can be reduced to a model-based equivalent, SPGP-TD. We perform comprehensive simulation studies and also conduct physical learning trials with an underwater robot. Our results show SPGP-SARSA can outperform the state-of-the-art sparse method, replicate the prediction quality of its exact counterpart, and be applied to solve underwater navigation tasks.
Cite
@article{arxiv.1810.01217,
title = {Sparse Gaussian Process Temporal Difference Learning for Marine Robot Navigation},
author = {John Martin and Jinkun Wang and Brendan Englot},
journal= {arXiv preprint arXiv:1810.01217},
year = {2018}
}
Comments
2018 Conference on Robot Learning (CoRL)