While recent vision-language models (VLMs) demonstrate strong image understanding, their ability to "think with images", i.e., to reason through multi-step visual interactions, remains limited. We introduce VISTA-Gym, a scalable training environment for incentivizing tool-integrated visual reasoning capabilities in VLMs. VISTA-Gym unifies diverse real-world multimodal reasoning tasks (7 tasks from 13 datasets in total) with a standardized interface for visual tools (e.g., grounding, parsing), executable interaction loops, verifiable feedback signals, and efficient trajectory logging, enabling visual agentic reinforcement learning at scale. While recent VLMs exhibit strong text-only reasoning, both proprietary and open-source models still struggle with tool selection, invocation, and coordination. With VISTA-Gym, we train VISTA-R1 to interleave tool-use with agentic reasoning via multi-turn trajectory sampling and end-to-end reinforcement learning. Extensive experiments across 11 public reasoning-intensive VQA benchmarks show that VISTA-R1-8B outperforms state-of-the-art baselines with similar sizes by 9.51%-18.72%, demonstrating VISTA-Gym as an effective training ground to unlock the tool-integrated reasoning capabilities for VLMs.
@article{arxiv.2511.19773,
title = {Scaling Agentic Reinforcement Learning for Tool-Integrated Reasoning in VLMs},
author = {Meng Lu and Ran Xu and Yi Fang and Wenxuan Zhang and Yue Yu and Gaurav Srivastava and Yuchen Zhuang and Mohamed Elhoseiny and Charles Fleming and Carl Yang and Zhengzhong Tu and Yang Xie and Guanghua Xiao and Hanrui Wang and Di Jin and Wenqi Shi and Xuan Wang},
journal= {arXiv preprint arXiv:2511.19773},
year = {2025}
}