Active learning from demonstration allows a robot to query a human for specific types of input to achieve efficient learning. Existing work has explored a variety of active query strategies; however, to our knowledge, none of these strategies directly minimize the performance risk of the policy the robot is learning. Utilizing recent advances in performance bounds for inverse reinforcement learning, we propose a risk-aware active inverse reinforcement learning algorithm that focuses active queries on areas of the state space with the potential for large generalization error. We show that risk-aware active learning outperforms standard active IRL approaches on gridworld, simulated driving, and table setting tasks, while also providing a performance-based stopping criterion that allows a robot to know when it has received enough demonstrations to safely perform a task.
@article{arxiv.1901.02161,
title = {Risk-Aware Active Inverse Reinforcement Learning},
author = {Daniel S. Brown and Yuchen Cui and Scott Niekum},
journal= {arXiv preprint arXiv:1901.02161},
year = {2019}
}
Comments
In proceedings of the 2nd Conference on Robot Learning (CoRL) 2018