We propose an actor-critic, model-free, and online Reinforcement Learning (RL) framework for continuous-state continuous-action Markov Decision Processes (MDPs) when the reward is highly sparse but encompasses a high-level temporal structure. We represent this temporal structure by a finite-state machine and construct an on-the-fly synchronised product with the MDP and the finite machine. The temporal structure acts as a guide for the RL agent within the product, where a modular Deep Deterministic Policy Gradient (DDPG) architecture is proposed to generate a low-level control policy. We evaluate our framework in a Mars rover experiment and we present the success rate of the synthesised policy.
@article{arxiv.1909.11591,
title = {Modular Deep Reinforcement Learning with Temporal Logic Specifications},
author = {Lim Zun Yuan and Mohammadhosein Hasanbeig and Alessandro Abate and Daniel Kroening},
journal= {arXiv preprint arXiv:1909.11591},
year = {2019}
}
Comments
arXiv admin note: text overlap with arXiv:1902.00778