SQT -- std $Q$-target
Machine Learning
2024-06-04 v3 Artificial Intelligence
Abstract
Std -target is a conservative, actor-critic, ensemble, -learning-based algorithm, which is based on a single key -formula: -networks standard deviation, which is an "uncertainty penalty", and, serves as a minimalistic solution to the problem of overestimation bias. We implement SQT on top of TD3/TD7 code and test it against the state-of-the-art (SOTA) actor-critic algorithms, DDPG, TD3 and TD7 on seven popular MuJoCo and Bullet tasks. Our results demonstrate SQT's -target formula superiority over TD3's -target formula as a conservative solution to overestimation bias in RL, while SQT shows a clear performance advantage on a wide margin over DDPG, TD3, and TD7 on all tasks.
Cite
@article{arxiv.2402.05950,
title = {SQT -- std $Q$-target},
author = {Nitsan Soffair and Dotan Di-Castro and Orly Avner and Shie Mannor},
journal= {arXiv preprint arXiv:2402.05950},
year = {2024}
}