Learning Model Predictive Controllers with Real-Time Attention for Real-World Navigation
Abstract
Despite decades of research, existing navigation systems still face real-world challenges when deployed in the wild, e.g., in cluttered home environments or in human-occupied public spaces. To address this, we present a new class of implicit control policies combining the benefits of imitation learning with the robust handling of system constraints from Model Predictive Control (MPC). Our approach, called Performer-MPC, uses a learned cost function parameterized by vision context embeddings provided by Performers -- a low-rank implicit-attention Transformer. We jointly train the cost function and construct the controller relying on it, effectively solving end-to-end the corresponding bi-level optimization problem. We show that the resulting policy improves standard MPC performance by leveraging a few expert demonstrations of the desired navigation behavior in different challenging real-world scenarios. Compared with a standard MPC policy, Performer-MPC achieves >40% better goal reached in cluttered environments and >65% better on social metrics when navigating around humans.
Cite
@article{arxiv.2209.10780,
title = {Learning Model Predictive Controllers with Real-Time Attention for Real-World Navigation},
author = {Xuesu Xiao and Tingnan Zhang and Krzysztof Choromanski and Edward Lee and Anthony Francis and Jake Varley and Stephen Tu and Sumeet Singh and Peng Xu and Fei Xia and Sven Mikael Persson and Dmitry Kalashnikov and Leila Takayama and Roy Frostig and Jie Tan and Carolina Parada and Vikas Sindhwani},
journal= {arXiv preprint arXiv:2209.10780},
year = {2022}
}