Interaction-limited Inverse Reinforcement Learning

Martin Troussard; Emmanuel Pignat; Parameswaran Kamalaruban; Sylvain Calinon; Volkan Cevher

Interaction-limited Inverse Reinforcement Learning

Machine Learning 2020-07-02 v1 Machine Learning

Authors: Martin Troussard , Emmanuel Pignat , Parameswaran Kamalaruban , Sylvain Calinon , Volkan Cevher

Abstract

This paper proposes an inverse reinforcement learning (IRL) framework to accelerate learning when the learner-teacher \textit{interaction} is \textit{limited} during training. Our setting is motivated by the realistic scenarios where a helpful teacher is not available or when the teacher cannot access the learning dynamics of the student. We present two different training strategies: Curriculum Inverse Reinforcement Learning (CIRL) covering the teacher's perspective, and Self-Paced Inverse Reinforcement Learning (SPIRL) focusing on the learner's perspective. Using experiments in simulations and experiments with a real robot learning a task from a human demonstrator, we show that our training strategies can allow a faster training than a random teacher for CIRL and than a batch learner for SPIRL.

Keywords

reinforcement learning reinforcement learning from human feedback imitation learning

Cite

@article{arxiv.2007.00425,
  title  = {Interaction-limited Inverse Reinforcement Learning},
  author = {Martin Troussard and Emmanuel Pignat and Parameswaran Kamalaruban and Sylvain Calinon and Volkan Cevher},
  journal= {arXiv preprint arXiv:2007.00425},
  year   = {2020}
}

Interaction-limited Inverse Reinforcement Learning

Abstract

Keywords

Cite

Related papers