SARSA(0) Reinforcement Learning over Fully Homomorphic Encryption
Abstract
We consider a cloud-based control architecture in which the local plants outsource the control synthesis task to the cloud. In particular, we consider a cloud-based reinforcement learning (RL), where updating the value function is outsourced to the cloud. To achieve confidentiality, we implement computations over Fully Homomorphic Encryption (FHE). We use a CKKS encryption scheme and a modified SARSA(0) reinforcement learning to incorporate the encryption-induced delays. We then give a convergence result for the delayed updated rule of SARSA(0) with a blocking mechanism. We finally present a numerical demonstration via implementing on a classical pole-balancing problem.
Cite
@article{arxiv.2002.00506,
title = {SARSA(0) Reinforcement Learning over Fully Homomorphic Encryption},
author = {Jihoon Suh and Takashi Tanaka},
journal= {arXiv preprint arXiv:2002.00506},
year = {2021}
}
Comments
7 pages, 2 figures, submitted to SICE ISCS 2021; replaced with supplementary paragraphs added and format change for the correct publishing medium