Quantum Natural Policy Gradients: Towards Sample-Efficient Reinforcement Learning
Abstract
Reinforcement learning is a growing field in AI with a lot of potential. Intelligent behavior is learned automatically through trial and error in interaction with the environment. However, this learning process is often costly. Using variational quantum circuits as function approximators potentially can reduce this cost. In order to implement this, we propose the quantum natural policy gradient (QNPG) algorithm -- a second-order gradient-based routine that takes advantage of an efficient approximation of the quantum Fisher information matrix. We experimentally demonstrate that QNPG outperforms first-order based training on Contextual Bandits environments regarding convergence speed and stability and moreover reduces the sample complexity. Furthermore, we provide evidence for the practical feasibility of our approach by training on a 12-qubit hardware device.
Cite
@article{arxiv.2304.13571,
title = {Quantum Natural Policy Gradients: Towards Sample-Efficient Reinforcement Learning},
author = {Nico Meyer and Daniel D. Scherer and Axel Plinge and Christopher Mutschler and Michael J. Hartmann},
journal= {arXiv preprint arXiv:2304.13571},
year = {2023}
}
Comments
Accepted to the 1st International Workshop on Quantum Machine Learning: From Foundations to Applications (QML@QCE 2023), Bellevue, Washington, USA. 6 pages, 4 figures, 1 table