凹-凸与背包设定下的约束 episodic 强化学习
机器学习
2021-06-08 v2 人工智能
数据结构与算法
机器学习
摘要
我们提出一种用于带约束的表格 episodic 强化学习的算法。我们提供了具有模块化分析且带有强理论保证的算法，适用于凹奖励与凸约束的设定，以及带硬约束（背包）的设定。先前大多数约束强化学习的工作局限于线性约束，其余工作要么聚焦于可行性问题，要么聚焦于单 episode 设定。我们的实验表明，在现有约束 episodic 环境中，所提算法显著优于这些方法。
引用
@article{arxiv.2006.05051,
title = {Constrained episodic reinforcement learning in concave-convex and knapsack settings},
author = {Kianté Brantley and Miroslav Dudik and Thodoris Lykouris and Sobhan Miryoosefi and Max Simchowitz and Aleksandrs Slivkins and Wen Sun},
journal= {arXiv preprint arXiv:2006.05051},
year = {2021}
}
备注
The NeurIPS 2020 version of this paper includes a small bug, leading to an incorrect dependence on H in Theorem 3.4. This version fixes it by adjusting Eq. (9), Theorem 3.4 and the relevant proofs. Changes in the main text are noted in red. Changes in the appendix are limited to Appendices B.1, B.5, and B.6 and the statement of Lemma F.3