Deploying humanoid robots to interact with real-world environments--such as carrying objects or sitting on chairs--requires generalizable, lifelike motions and robust scene perception. Although prior approaches have advanced each capability individually, combining them in a unified system is still an ongoing challenge. In this work, we present a physical-world humanoid-scene interaction system, PhysHSI, that enables humanoids to autonomously perform diverse interaction tasks while maintaining natural and lifelike behaviors. PhysHSI comprises a simulation training pipeline and a real-world deployment system. In simulation, we adopt adversarial motion prior-based policy learning to imitate natural humanoid-scene interaction data across diverse scenarios, achieving both generalization and lifelike behaviors. For real-world deployment, we introduce a coarse-to-fine object localization module that combines LiDAR and camera inputs to provide continuous and robust scene perception. We validate PhysHSI on four representative interactive tasks--box carrying, sitting, lying, and standing up--in both simulation and real-world settings, demonstrating consistently high success rates, strong generalization across diverse task goals, and natural motion patterns.
@article{arxiv.2510.11072,
title = {PhysHSI: Towards a Real-World Generalizable and Natural Humanoid-Scene Interaction System},
author = {Huayi Wang and Wentao Zhang and Runyi Yu and Tao Huang and Junli Ren and Feiyu Jia and Zirui Wang and Xiaojie Niu and Xiao Chen and Jiahe Chen and Qifeng Chen and Jingbo Wang and Jiangmiao Pang},
journal= {arXiv preprint arXiv:2510.11072},
year = {2025}
}