$f$-GAIL: Learning $f$-Divergence for Generative Adversarial Imitation Learning
Abstract
Imitation learning (IL) aims to learn a policy from expert demonstrations that minimizes the discrepancy between the learner and expert behaviors. Various imitation learning algorithms have been proposed with different pre-determined divergences to quantify the discrepancy. This naturally gives rise to the following question: Given a set of expert demonstrations, which divergence can recover the expert policy more accurately with higher data efficiency? In this work, we propose -GAIL, a new generative adversarial imitation learning (GAIL) model, that automatically learns a discrepancy measure from the -divergence family as well as a policy capable of producing expert-like behaviors. Compared with IL baselines with various predefined divergence measures, -GAIL learns better policies with higher data efficiency in six physics-based control tasks.
Cite
@article{arxiv.2010.01207,
title = {$f$-GAIL: Learning $f$-Divergence for Generative Adversarial Imitation Learning},
author = {Xin Zhang and Yanhua Li and Ziming Zhang and Zhi-Li Zhang},
journal= {arXiv preprint arXiv:2010.01207},
year = {2020}
}