SimWorld: An Open-ended Realistic Simulator for Autonomous Agents in Physical and Social Worlds
Abstract
While LLM/VLM-powered AI agents have advanced rapidly in math, coding, and computer use, their applications in complex physical and social environments remain challenging. Building agents that can survive and thrive in the real world (for example, by autonomously earning income or running a business) requires massive-scale interaction, reasoning, training, and evaluation across diverse embodied scenarios. However, existing world simulators for such development fall short: they often rely on limited hand-crafted environments, simulate simplified game-like physics and social rules, and lack native support for LLM/VLM agents. We introduce SimWorld, a new simulator built on Unreal Engine 5, designed for developing and evaluating LLM/VLM agents in rich, real-world-like settings. SimWorld offers three core capabilities: (1) realistic, open-ended world simulation, including accurate physical and social dynamics and language-driven procedural environment generation; (2) a rich interface for LLM/VLM agents, with multimodal world inputs and open-vocabulary actions at varying levels of abstraction; and (3) diverse and extensible physical and social reasoning scenarios that are easily customizable by users. We demonstrate SimWorld by deploying frontier LLM agents (e.g., GPT-4o, Gemini-2.5-Flash, Claude-3.5, and DeepSeek-Prover-V2) on long-horizon multi-agent delivery tasks involving strategic cooperation and competition. The results reveal distinct reasoning patterns and limitations across models. We open-source SimWorld and hope it becomes a foundational platform for advancing real-world agent intelligence across disciplines: https://simworld.org.
Cite
@article{arxiv.2512.01078,
title = {SimWorld: An Open-ended Realistic Simulator for Autonomous Agents in Physical and Social Worlds},
author = {Jiawei Ren and Yan Zhuang and Xiaokang Ye and Lingjun Mao and Xuhong He and Jianzhi Shen and Mrinaal Dogra and Yiming Liang and Ruixuan Zhang and Tianai Yue and Yiqing Yang and Eric Liu and Ryan Wu and Kevin Benavente and Rajiv Mandya Nagaraju and Muhammad Faayez and Xiyan Zhang and Dhruv Vivek Sharma and Xianrui Zhong and Ziqiao Ma and Tianmin Shu and Zhiting Hu and Lianhui Qin},
journal= {arXiv preprint arXiv:2512.01078},
year = {2026}
}