Incorporating Voice Instructions in Model-Based Reinforcement Learning for Self-Driving Cars
Abstract
This paper presents a novel approach that supports natural language voice instructions to guide deep reinforcement learning (DRL) algorithms when training self-driving cars. DRL methods are popular approaches for autonomous vehicle (AV) agents. However, most existing methods are sample- and time-inefficient and lack a natural communication channel with the human expert. In this paper, how new human drivers learn from human coaches motivates us to study new ways of human-in-the-loop learning and a more natural and approachable training interface for the agents. We propose incorporating natural language voice instructions (NLI) in model-based deep reinforcement learning to train self-driving cars. We evaluate the proposed method together with a few state-of-the-art DRL methods in the CARLA simulator. The results show that NLI can help ease the training process and significantly boost the agents' learning speed.
Keywords
Cite
@article{arxiv.2206.10249,
title = {Incorporating Voice Instructions in Model-Based Reinforcement Learning for Self-Driving Cars},
author = {Mingze Wang and Ziyang Zhang and Grace Hui Yang},
journal= {arXiv preprint arXiv:2206.10249},
year = {2022}
}
Comments
NeurIPS 2021 Workshop on Machine Learning for Autonomous Driving