We propose a framework to enable multipurpose assistive mobile robots to autonomously wipe tables to clean spills and crumbs. This problem is challenging, as it requires planning wiping actions while reasoning over uncertain latent dynamics of crumbs and spills captured via high-dimensional visual observations. Simultaneously, we must guarantee constraints satisfaction to enable safe deployment in unstructured cluttered environments. To tackle this problem, we first propose a stochastic differential equation to model crumbs and spill dynamics and absorption with a robot wiper. Using this model, we train a vision-based policy for planning wiping actions in simulation using reinforcement learning (RL). To enable zero-shot sim-to-real deployment, we dovetail the RL policy with a whole-body trajectory optimization framework to compute base and arm joint trajectories that execute the desired wiping motions while guaranteeing constraints satisfaction. We extensively validate our approach in simulation and on hardware. Video: https://youtu.be/inORKP4F3EI
@article{arxiv.2210.10865,
title = {Robotic Table Wiping via Reinforcement Learning and Whole-body Trajectory Optimization},
author = {Thomas Lew and Sumeet Singh and Mario Prats and Jeffrey Bingham and Jonathan Weisz and Benjie Holson and Xiaohan Zhang and Vikas Sindhwani and Yao Lu and Fei Xia and Peng Xu and Tingnan Zhang and Jie Tan and Montserrat Gonzalez},
journal= {arXiv preprint arXiv:2210.10865},
year = {2022}
}