A Note on Stability in Asynchronous Stochastic Approximation without Communication Delays
Machine Learning
2024-08-15 v2 Optimization and Control
Abstract
In this paper, we study asynchronous stochastic approximation algorithms without communication delays. Our main contribution is a stability proof for these algorithms that extends a method of Borkar and Meyn by accommodating more general noise conditions. We also derive convergence results from this stability result and discuss their application in important average-reward reinforcement learning problems.
Cite
@article{arxiv.2312.15091,
title = {A Note on Stability in Asynchronous Stochastic Approximation without Communication Delays},
author = {Huizhen Yu and Yi Wan and Richard S. Sutton},
journal= {arXiv preprint arXiv:2312.15091},
year = {2024}
}
Comments
Corrected typos and a minor error; parts of this material will be included in a separate future arXiv preprint