English

Beyond Text Prompts: Visual-to-Visual Generation as A Unified Paradigm

Computer Vision and Pattern Recognition 2026-05-27 v2

Abstract

Humans often specify and create through visual artifacts: typography sheets, sketches, reference images, and annotated scenes. Yet modern visual generators still ask users to serialize this intent into text, a bottleneck that compresses signals like spatial structure, exact appearance, and glyph shape. We propose \textbf{\emph{visual-to-visual} (V2V)} generation, in which the user conditions a generative model with a visual specification page rather than a text prompt. The page is not an edit target, but a visual document that specifies the desired output. We introduce \textbf{V2V-Zero}, a training-free framework that exposes this interface in existing vision-language model (VLM) conditioned generators by replacing text-only conditioning with final-layer hidden states extracted from visual pages, exploiting the fact that the frozen VLM already maps both text and images into the generator's conditioning space. On GenEval, V2V-Zero reaches 0.85 with a frozen Qwen-Image backbone, closely matching its optimized text-to-image performance without fine-tuning. To evaluate the broader V2V space, we introduce \textbf{Simple-V2V Bench}, spanning seven visual-conditioning tasks and seven models, including GPT Image 2, Nano Banana 2, Seedream 5.0 Lite, open-weight baselines, and a video extension. V2V-Zero scores 32.7/100, outperforming evaluated open-weight image baselines and revealing a clear capability hierarchy: attribute binding is strong, content generation is unreliable, and structural control remains hard even for commercial systems. A HunyuanVideo-1.5 extension scores 20.2/100, showing the interface transfers beyond images. Mechanistic analysis shows the default reasoning path is primarily visually routed, with 95.0\% of conditioning-token attention mass on visual-page hidden states.

Keywords

Cite

@article{arxiv.2605.12271,
  title  = {Beyond Text Prompts: Visual-to-Visual Generation as A Unified Paradigm},
  author = {Yaofang Liu and Kangning Cui and Meng Chu and Zhaoqing Li and Suiyun Zhang and Jean-Michel Morel and Xiaodong Cun and Haoxuan Che and Rui Liu and Raymond H. Chan},
  journal= {arXiv preprint arXiv:2605.12271},
  year   = {2026}
}

Comments

Project Page: https://yaofang-liu.github.io/V2V_Web

R2 v1 2026-07-22T07:07:57.501Z