通过近似信息状态实现抽象 Sim2Real

近年来，强化学习 (RL) 在机器人领域展现出惊人的成功，尤其是在给定快速且准确的仿真器用于特定任务时。当使用 RL 和仿真时，更多的仿真真实性通常是有益的，但随着机器人在日益复杂和广泛的域中部署，这种做法变得更加困难。在此类设置下，仿真器很可能会未能建模给定目标任务的所有相关细节，这一观察结果激励了研究在仿真器中省略关键任务细节的情况下的 sim2real。In this paper, we formalize and study the abstract sim2real problem: given an abstract simulator that models a target task at a coarse level of abstraction, how can we train a policy with RL in the abstract simulator and successfully transfer it to the real-world? Our first contribution is to formalize this problem using the language of state abstraction from the RL literature. This framing shows that an abstract simulator can be grounded to match the target task if the grounded abstract dynamics take the history of states into account. Based on the formalism, we then introduce a method that uses real-world task data to correct the dynamics of the abstract simulator. We then show that this method enables successful policy transfer both in sim2sim and sim2real evaluation.。