Warm-Start Variational Quantum Policy Iteration
Abstract
Reinforcement learning is a powerful framework aiming to determine optimal behavior in highly complex decision-making scenarios. This objective can be achieved using policy iteration, which requires to solve a typically large linear system of equations. We propose the variational quantum policy iteration (VarQPI) algorithm, realizing this step with a NISQ-compatible quantum-enhanced subroutine. Its scalability is supported by an analysis of the structure of generic reinforcement learning environments, laying the foundation for potential quantum advantage with utility-scale quantum computers. Furthermore, we introduce the warm-start initialization variant (WS-VarQPI) that significantly reduces resource overhead. The algorithm solves a large FrozenLake environment with an underlying 256x256-dimensional linear system, indicating its practical robustness.
Keywords
Cite
@article{arxiv.2404.10546,
title = {Warm-Start Variational Quantum Policy Iteration},
author = {Nico Meyer and Jakob Murauer and Alexander Popov and Christian Ufrecht and Axel Plinge and Christopher Mutschler and Daniel D. Scherer},
journal= {arXiv preprint arXiv:2404.10546},
year = {2025}
}
Comments
Accepted to the IEEE International Conference on Quantum Computing and Engineering (QCE 2024), Montr\'eal, Qu\'ebec, Canada. 9 pages, 6 figures, 1 table