English

Graph Diffusion Transformers are In-Context Molecular Designers

Machine Learning 2025-10-13 v1 Artificial Intelligence

Abstract

In-context learning allows large models to adapt to new tasks from a few demonstrations, but it has shown limited success in molecular design. Existing databases such as ChEMBL contain molecular properties spanning millions of biological assays, yet labeled data for each property remain scarce. To address this limitation, we introduce demonstration-conditioned diffusion models (DemoDiff), which define task contexts using a small set of molecule-score examples instead of text descriptions. These demonstrations guide a denoising Transformer to generate molecules aligned with target properties. For scalable pretraining, we develop a new molecular tokenizer with Node Pair Encoding that represents molecules at the motif level, requiring 5.5×\times fewer nodes. We curate a dataset containing millions of context tasks from multiple sources covering both drugs and materials, and pretrain a 0.7-billion-parameter model on it. Across 33 design tasks in six categories, DemoDiff matches or surpasses language models 100-1000×\times larger and achieves an average rank of 3.63 compared to 5.25-10.20 for domain-specific approaches. These results position DemoDiff as a molecular foundation model for in-context molecular design. Our code is available at https://github.com/liugangcode/DemoDiff.

Keywords

Cite

@article{arxiv.2510.08744,
  title  = {Graph Diffusion Transformers are In-Context Molecular Designers},
  author = {Gang Liu and Jie Chen and Yihan Zhu and Michael Sun and Tengfei Luo and Nitesh V Chawla and Meng Jiang},
  journal= {arXiv preprint arXiv:2510.08744},
  year   = {2025}
}

Comments

29 pages, 16 figures, 17 tables. Model available at: https://huggingface.co/liuganghuggingface/DemoDiff-0.7B

R2 v1 2026-07-01T06:27:59.233Z