English

Beyond Simply Environment Scaling: Designing Effective Environment Distributions for Multimodal Agent Learning

Computer Vision and Pattern Recognition 2026-08-04 v1

Abstract

Recent works train agents by constructing large-scale multimodal environment pools. However, we find that simply increasing the number of multimodal environments does not always benefit. We further analyze the limitations in current multimodal environment distributions through a series of experiments. Based on these findings, we study how to build more effective training environment distributions from two dimensions: **diversity** and **difficulty structure**. For diversity, we propose **Ability-aware Environment Selection (AES)** to obtain diverse environment sets. For difficulty structure, we propose **Hierarchical Difficulty Curriculum (HDC)**, which organizes curriculum learning through two difficulty levels: harness weakening and state-scale progression. Experiments show that AES and HDC effectively improve multimodal agent training.

Cite

@article{arxiv.2608.03571,
  title  = {Beyond Simply Environment Scaling: Designing Effective Environment Distributions for Multimodal Agent Learning},
  author = {Kejian Zhu and Zhuoran Jin and Dongqi Huang and Hongbang Yuan and Yupu Hao and Kang Liu and Jun Zhao},
  journal= {arXiv preprint arXiv:2608.03571},
  year   = {2026}
}

Comments

Code: https://github.com/GaryStack/Beyond-MMEnv-Scaling