English

I2E: From Image Pixels to Actionable Interactive Environments for Text-Guided Image Editing

Computer Vision and Pattern Recognition 2026-04-08 v2

Abstract

Existing text-guided image editing methods primarily rely on end-to-end pixel-level inpainting paradigm. Despite its success in simple scenarios, this paradigm still significantly struggles with compositional editing tasks that require precise local control and complex multi-object spatial reasoning. This paradigm is severely limited by 1) the implicit coupling of planning and execution, 2) the lack of object-level control granularity, and 3) the reliance on unstructured, pixel-centric modeling. To address these limitations, we propose I2E, a novel "Decompose-then-Action" paradigm that revisits image editing as an actionable interaction process within a structured environment. I2E utilizes a Decomposer to transform unstructured images into discrete, manipulable object layers and then introduces a physics-aware Vision-Language-Action Agent to parse complex instructions into a series of atomic actions via Chain-of-Thought reasoning. Further, we also construct I2E-Bench, a benchmark designed for multi-instance spatial reasoning and high-precision editing. Experimental results on I2E-Bench and multiple public benchmarks demonstrate that I2E significantly outperforms state-of-the-art methods in handling complex compositional instructions, maintaining physical plausibility, and ensuring multi-turn editing stability.

Keywords

Cite

@article{arxiv.2601.03741,
  title  = {I2E: From Image Pixels to Actionable Interactive Environments for Text-Guided Image Editing},
  author = {Jinghan Yu and Junhao Xiao and Chenyu Zhu and Jiaming Li and Jia Li and HanMing Deng and Xirui Wang and Guoli Jia and Jianjun Li and Xiang Bai and Bowen Zhou and Zhiyuan Ma},
  journal= {arXiv preprint arXiv:2601.03741},
  year   = {2026}
}
R2 v1 2026-07-01T08:54:00.492Z