English
Related papers

Related papers: Motion Guidance: Diffusion-Based Image Editing wit…

200 papers

Text-driven image and video diffusion models have recently achieved unprecedented generation realism. While diffusion models have been successfully applied for image editing, very few works have done so for video editing. We present the…

Computer Vision and Pattern Recognition · Computer Science 2023-02-03 Eyal Molad , Eliahu Horwitz , Dani Valevski , Alex Rav Acha , Yossi Matias , Yael Pritch , Yaniv Leviathan , Yedid Hoshen

Research in vision-language models has seen rapid developments off-late, enabling natural language-based interfaces for image generation and manipulation. Many existing text guided manipulation techniques are restricted to specific classes…

Computer Vision and Pattern Recognition · Computer Science 2024-05-07 Paramanand Chandramouli , Kanchana Vaishnavi Gandikota

Adding additional control to pretrained diffusion models has become an increasingly popular research area, with extensive applications in computer vision, reinforcement learning, and AI for science. Recently, several studies have proposed…

Machine Learning · Computer Science 2024-05-30 Yifei Shen , Xinyang Jiang , Yezhen Wang , Yifan Yang , Dongqi Han , Dongsheng Li

Diffusion models equipped with language models demonstrate excellent controllability in image generation tasks, allowing image processing to adhere to human instructions. However, the lack of diverse instruction-following data hampers the…

Computer Vision and Pattern Recognition · Computer Science 2024-10-11 Yongsheng Yu , Ziyun Zeng , Hang Hua , Jianlong Fu , Jiebo Luo

Despite recent advances in large-scale text-to-image generative models, manipulating real images with these models remains a challenging problem. The main limitations of existing editing methods are that they either fail to perform with…

Computer Vision and Pattern Recognition · Computer Science 2024-09-26 Vadim Titov , Madina Khalmatova , Alexandra Ivanova , Dmitry Vetrov , Aibek Alanov

When using a diffusion model for image editing, there are times when the modified image can differ greatly from the source. To address this, we apply a dual-guidance approach to maintain high fidelity to the original in areas that are not…

Computer Vision and Pattern Recognition · Computer Science 2023-12-13 Ruichen Zhang

Diffusion-based image editing is a composite process of preserving the source image content and generating new content or applying modifications. While current editing approaches have made improvements under text guidance, most of them have…

Computer Vision and Pattern Recognition · Computer Science 2024-03-18 Tianrui Huang , Pu Cao , Lu Yang , Chun Liu , Mengjie Hu , Zhiwei Liu , Qing Song

Recent large-scale pre-trained diffusion models have demonstrated a powerful generative ability to produce high-quality videos from detailed text descriptions. However, exerting control over the motion of objects in videos generated by any…

Computer Vision and Pattern Recognition · Computer Science 2025-01-09 Changgu Chen , Junwei Shu , Gaoqi He , Changbo Wang , Yang Li

Despite the great success of large-scale text-to-image diffusion models in image generation and image editing, existing methods still struggle to edit the layout of real images. Although a few works have been proposed to tackle this…

Computer Vision and Pattern Recognition · Computer Science 2025-03-11 Tao Xia , Yudi Zhang , Ting Liu Lei Zhang

Existing diffusion-based video editing models have made gorgeous advances for editing attributes of a source video over time but struggle to manipulate the motion information while preserving the original protagonist's appearance and…

Computer Vision and Pattern Recognition · Computer Science 2023-12-01 Shuyuan Tu , Qi Dai , Zhi-Qi Cheng , Han Hu , Xintong Han , Zuxuan Wu , Yu-Gang Jiang

Recent text-guided diffusion models provide powerful image generation capabilities. Currently, a massive effort is given to enable the modification of these images using text only as means to offer intuitive and versatile editing. To edit a…

Computer Vision and Pattern Recognition · Computer Science 2022-11-18 Ron Mokady , Amir Hertz , Kfir Aberman , Yael Pritch , Daniel Cohen-Or

Recent advancements in diffusion and flow-matching models have demonstrated remarkable capabilities in high-fidelity image synthesis. A prominent line of research involves reward-guided guidance, which steers the generation process during…

Computer Vision and Pattern Recognition · Computer Science 2026-05-01 Jinho Chang , Jaemin Kim , Jong Chul Ye

Denoising diffusion models hold great promise for generating diverse and realistic human motions. However, existing motion diffusion models largely disregard the laws of physics in the diffusion process and often generate…

Computer Vision and Pattern Recognition · Computer Science 2023-08-22 Ye Yuan , Jiaming Song , Umar Iqbal , Arash Vahdat , Jan Kautz

The recent wave of large-scale text-to-image diffusion models has dramatically increased our text-based image generation abilities. These models can generate realistic images for a staggering variety of prompts and exhibit impressive…

Machine Learning · Computer Science 2023-09-14 Alexander C. Li , Mihir Prabhudesai , Shivam Duggal , Ellis Brown , Deepak Pathak

Diffusion models have revolutionized image generation and editing, producing state-of-the-art results in conditioned and unconditioned image synthesis. While current techniques enable user control over the degree of change in an image edit,…

Computer Vision and Pattern Recognition · Computer Science 2024-03-01 Eran Levin , Ohad Fried

Pose and body shape editing in a human image has received increasing attention. However, current methods often struggle with dataset biases and deteriorate realism and the person's identity when users make large edits. We propose a one-shot…

Computer Vision and Pattern Recognition · Computer Science 2024-01-09 Yuta Okuyama , Yuki Endo , Yoshihiro Kanamori

Recent advances in diffusion models enable many powerful instruments for image editing. One of these instruments is text-driven image manipulations: editing semantic attributes of an image according to the provided text description. %…

Computer Vision and Pattern Recognition · Computer Science 2023-04-11 Nikita Starodubcev , Dmitry Baranchuk , Valentin Khrulkov , Artem Babenko

The use of denoising diffusion models is becoming increasingly popular in the field of image editing. However, current approaches often rely on either image-guided methods, which provide a visual reference but lack control over semantic…

Computer Vision and Pattern Recognition · Computer Science 2025-01-07 Zhanbo Feng , Zenan Ling , Xinyu Lu , Ci Gong , Feng Zhou , Wugedele Bao , Jie Li , Fan Yang , Robert C. Qiu

Existing diffusion-based methods have achieved impressive results in human motion editing. However, these methods often exhibit significant ghosting and body distortion in unseen in-the-wild cases. In this paper, we introduce…

Computer Vision and Pattern Recognition · Computer Science 2024-10-16 Yi Zuo , Lingling Li , Licheng Jiao , Fang Liu , Xu Liu , Wenping Ma , Shuyuan Yang , Yuwei Guo

With the rise of large, publicly-available text-to-image diffusion models, text-guided real image editing has garnered much research attention recently. Existing methods tend to either rely on some form of per-instance or per-task…

Computer Vision and Pattern Recognition · Computer Science 2022-11-16 Adham Elarabawy , Harish Kamath , Samuel Denton