English

CoEditor++: Instruction-based Visual Editing via Cognitive Reasoning

Human-Computer Interaction 2026-03-09 v1 Computer Vision and Pattern Recognition

Abstract

Recent advances in large multimodal models (LMMs) have enabled instruction-based image editing, allowing users to modify visual content via natural language descriptions. However, existing approaches often struggle with high-level semantic reasoning and visual consistency, particularly under ambiguous or complex instructions. To address these challenges, we propose CoEditor++, a cognitively structured, training-free framework that decomposes editing into "what to edit" and "how to edit" through two cognitive stages with a reflective self-selection mechanism, enabling robust, fine-grained, and interpretable editing. Built entirely from open-sourced components, CoEditor++ requires no additional training or fine-tuning, ensuring transparency and cross-domain applicability. We evaluate CoEditor++ on SmartEdit, a widely used benchmark for general editing, and AltBear, a privacy and compliance-oriented benchmark. Experimental results show that CoEditor++ achieves state-of-the-art performance in both general editing and responsible editing tasks compared with open-sourced models that require training on specialized editing datasets maintaining significantly higher visual consistency. When compared with closed-source models such as Nano Banana Pro or GPT-4o, CoEditor++ preserves comparable instruction following while still substantially outperforming them in visual consistency. Extensive ablation studies confirm that the effectiveness of CoEditor++ benefits from its structured cognitive design rather than any specific model component. Our findings suggest the potential toward cognitive-centric instruction-based image editing.

Keywords

Cite

@article{arxiv.2603.05518,
  title  = {CoEditor++: Instruction-based Visual Editing via Cognitive Reasoning},
  author = {Minheng Ni and Yutao Fan and Zhengyuan Yang and Yeli Shen and Yuxiang Wei and Yaowen Zhang and Lijuan Wang and Lei Zhang and Wangmeng Zuo},
  journal= {arXiv preprint arXiv:2603.05518},
  year   = {2026}
}
R2 v1 2026-07-01T11:05:30.191Z