English

OrbiSim: World Models as Differentiable Physics Engines for Embodied Intelligence

Robotics 2026-05-19 v1 Machine Learning

Abstract

We present OrbiSim, a novel robotic simulation paradigm that redefines world models as a fully differentiable physics engine for embodied intelligence. Unlike prior world models that focus on unconstrained imagination in latent or visual domains, OrbiSim establishes a unified, physically-grounded pathway that bridges structured scene assets, neural dynamics, and downstream reinforcement learning. By enabling end-to-end differentiability throughout the entire simulation loop -- spanning from explicit state transitions to visual observation generation -- OrbiSim supports tasks traditionally intractable for classical simulators, such as differentiable contact modeling, gradient-based policy optimization under sparse rewards, and intuitive physical inference. Empirical results demonstrate that OrbiSim significantly outperforms state-of-the-art world models in both predictive fidelity and control performance. Furthermore, its consistent responsiveness to asset configurations and physical parameters suggests its potential as a differentiable tool for enhancing robot simulation and policy training.

Keywords

Cite

@article{arxiv.2605.16395,
  title  = {OrbiSim: World Models as Differentiable Physics Engines for Embodied Intelligence},
  author = {Jiajian Li and Jingyuan Huang and Junru Gong and Qi Wang and Xiaokang Yang and Yunbo Wang},
  journal= {arXiv preprint arXiv:2605.16395},
  year   = {2026}
}

Comments

Project page: https://jjleejj85.github.io/projects/orbisim