English
Related papers

Related papers: Composition and Alignment of Diffusion Models usin…

200 papers

Diffusion models have made substantial advances in image generation, yet models trained on large, unfiltered datasets often yield outputs misaligned with human preferences. Numerous methods have been proposed to fine-tune pre-trained…

Computer Vision and Pattern Recognition · Computer Science 2025-05-19 Fu-Yun Wang , Yunhao Shui , Jingtan Piao , Keqiang Sun , Hongsheng Li

Recently, diffusion models have been used to solve various inverse problems in an unsupervised manner with appropriate modifications to the sampling process. However, the current solvers, which recursively apply a reverse diffusion step…

Machine Learning · Computer Science 2024-05-21 Hyungjin Chung , Byeongsu Sim , Dohoon Ryu , Jong Chul Ye

Guided or controlled data generation with diffusion models\blfootnote{Partial preliminary results of this work appeared in International Conference on Machine Learning 2025 \citep{li2025provable}.} has become a cornerstone of modern…

Machine Learning · Statistics 2025-12-05 Yuchen Jiao , Yuxin Chen , Gen Li

Text-to-image models produce images that align well with natural language prompts, but compositional generation has long been a central challenge. Models often struggle to satisfy multiple concepts within a single prompt, frequently…

Artificial Intelligence · Computer Science 2026-03-20 Jungmyung Wi , Hyunsoo Kim , Donghyun Kim

Diffusion models (DMs) are a powerful generative framework that have attracted significant attention in recent years. However, the high computational cost of training DMs limits their practical applications. In this paper, we start with a…

Machine Learning · Computer Science 2024-04-12 Tianshuo Xu , Peng Mi , Ruilin Wang , Yingcong Chen

Model-based reinforcement learning methods often use learning only for the purpose of estimating an approximate dynamics model, offloading the rest of the decision-making work to classical trajectory optimizers. While conceptually simple,…

Machine Learning · Computer Science 2022-12-22 Michael Janner , Yilun Du , Joshua B. Tenenbaum , Sergey Levine

Conditional diffusion probabilistic models can model the distribution of natural images and can generate diverse and realistic samples based on given conditions. However, oftentimes their results can be unrealistic with observable color…

Computer Vision and Pattern Recognition · Computer Science 2022-12-15 Kangfu Mei , Nithin Gopalakrishnan Nair , Vishal M. Patel

In this work, we focus on the alignment problem of diffusion models with a continuous reward function, which represents specific objectives for downstream tasks, such as increasing darkness or improving the aesthetics of images. The central…

Machine Learning · Computer Science 2024-10-03 Zhiwei Tang , Jiangweizhi Peng , Jiasheng Tang , Mingyi Hong , Fan Wang , Tsung-Hui Chang

As Diffusion Models have shown promising performance, a lot of efforts have been made to improve the controllability of Diffusion Models. However, how to train Diffusion Models to have the disentangled latent spaces and how to naturally…

Computer Vision and Pattern Recognition · Computer Science 2025-07-02 Wonwoong Cho , Hareesh Ravi , Midhun Harikumar , Vinh Khuc , Krishna Kumar Singh , Jingwan Lu , David I. Inouye , Ajinkya Kale

Sequential decision-making is desired to align with human intents and exhibit versatility across various tasks. Previous methods formulate it as a conditional generation process, utilizing return-conditioned diffusion models to directly…

Machine Learning · Computer Science 2024-10-11 Xudong Yu , Chenjia Bai , Haoran He , Changhong Wang , Xuelong Li

Controllable layout generation refers to the process of creating a plausible visual arrangement of elements within a graphic design (e.g., document and web designs) with constraints representing design intentions. Although recent…

Computer Vision and Pattern Recognition · Computer Science 2024-05-17 Jian Chen , Ruiyi Zhang , Yufan Zhou , Rajiv Jain , Zhiqiang Xu , Ryan Rossi , Changyou Chen

In this paper we describe a novel framework for diffusion-based generative modeling on constrained spaces. In particular, we introduce manual bridges, a framework that expands the kinds of constraints that can be practically used to form…

Machine Learning · Computer Science 2025-02-28 Saeid Naderiparizi , Xiaoxuan Liang , Berend Zwartsenberg , Frank Wood

Consistency models have been proposed for fast generative modeling, achieving results competitive with diffusion and flow models. However, these methods exhibit inherent instability and limited reproducibility when training from scratch,…

Machine Learning · Computer Science 2026-02-02 Youngjoong Kim , Duhoe Kim , Woosung Kim , Jaesik Park

Diffusion-based planning, learning, and control methods present a promising branch of powerful and expressive decision-making solutions. Given the growing interest, such methods have undergone numerous refinements over the past years.…

Machine Learning · Computer Science 2025-02-19 Dom Huh , Prasant Mohapatra

Learning from feedback has been shown to enhance the alignment between text prompts and images in text-to-image diffusion models. However, due to the lack of focus in feedback content, especially regarding the object type and quantity,…

Computer Vision and Pattern Recognition · Computer Science 2024-12-03 Xuexiang Niu , Jinping Tang , Lei Wang , Ge Zhu

Diffusion Models represent a significant advancement in generative modeling, employing a dual-phase process that first degrades domain-specific information via Gaussian noise and restores it through a trainable model. This framework enables…

Neural and Evolutionary Computing · Computer Science 2024-11-21 Benedikt Hartl , Yanbo Zhang , Hananel Hazan , Michael Levin

Post-training alignment of diffusion models relies on simplified signals, such as scalar rewards or binary preferences. This limits alignment with complex human expertise, which is hierarchical and fine-grained. To address this, we first…

Computer Vision and Pattern Recognition · Computer Science 2026-01-09 Chenye Meng , Zejian Li , Zhongni Liu , Yize Li , Changle Xie , Kaixin Jia , Ling Yang , Huanghuang Deng , Shiying Ding , Shengyuan Zhang , Jiayi Li , Lingyun Sun

Recent advancements in diffusion models (DMs) have been propelled by alignment methods that post-train models to better conform to human preferences. However, these approaches typically require computation-intensive training of a base model…

Computer Vision and Pattern Recognition · Computer Science 2025-08-05 Zejian Li , Yize Li , Chenye Meng , Zhongni Liu , Yang Ling , Shengyuan Zhang , Guang Yang , Changyuan Yang , Zhiyuan Yang , Lingyun Sun

In recent years, diffusion models have gained popularity for their ability to generate higher-quality images in comparison to GAN models. However, like any other large generative models, these models require a huge amount of data,…

Computer Vision and Pattern Recognition · Computer Science 2023-12-21 Rajesh Shrestha , Bowen Xie

Diffusion models have been successfully adapted to text generation tasks by mapping the discrete text into the continuous space. However, there exist nonnegligible gaps between training and inference, owing to the absence of the forward…

Computation and Language · Computer Science 2023-05-09 Zecheng Tang , Pinzheng Wang , Keyan Zhou , Juntao Li , Ziqiang Cao , Min Zhang