English

TAUE: Training-free Noise Transplant and Cultivation Diffusion Model

Computer Vision and Pattern Recognition 2026-03-18 v2 Artificial Intelligence Graphics Machine Learning

Abstract

Despite the remarkable success of text-to-image diffusion models, their output of a single, flattened image remains a critical bottleneck for professional applications requiring layer-wise control. Existing solutions either rely on fine-tuning with large, inaccessible datasets or are training-free yet limited to generating isolated foreground elements, failing to produce a complete and coherent scene. To address this, we introduce the Training-free Noise Transplantation and Cultivation Diffusion Model (TAUE), a novel framework for layer-wise image generation that requires neither fine-tuning nor additional data. TAUE embeds global structural information from intermediate denoising latents into the initial noise to preserve spatial coherence, and integrates semantic cues through cross-layer attention sharing to maintain contextual and visual consistency across layers. Extensive experiments demonstrate that TAUE achieves state-of-the-art performance among training-free methods, delivering image quality comparable to fine-tuned models while improving inter-layer consistency. Moreover, it enables new applications, such as layout-aware editing, multi-object composition, and background replacement, indicating potential for interactive, layer-separated generation systems in real-world creative workflows.

Keywords

Cite

@article{arxiv.2511.02580,
  title  = {TAUE: Training-free Noise Transplant and Cultivation Diffusion Model},
  author = {Daichi Nagai and Ryugo Morita and Shunsuke Kitada and Hitoshi Iyatomi},
  journal= {arXiv preprint arXiv:2511.02580},
  year   = {2026}
}

Comments

Accepted to CVPR 2026 Findings. The first two authors contributed equally. Project Page: https://iyatomilab.github.io/TAUE