Empowering large language models (LLMs) with chemical intelligence remains a challenge due to the scarcity of high-quality, domain-specific instruction-response datasets and the misalignment of existing synthetic data generation pipelines with the inherently hierarchical and rule-governed structure of chemical information. To address this, we propose ChemOrch, a framework that synthesizes chemically grounded instruction-response pairs through a two-stage process: task-controlled instruction generation and tool-aware response construction. ChemOrch enables controllable diversity and levels of difficulty for the generated tasks, and ensures response precision through tool planning and distillation, and tool-based self-repair mechanisms. The effectiveness of ChemOrch is evaluated based on: 1) the high quality of generated instruction data, demonstrating superior diversity and strong alignment with chemical constraints; 2) the reliable generation of evaluation tasks that more effectively reveal LLM weaknesses in chemistry; and 3) the significant improvement of LLM chemistry capabilities when the generated instruction data are used for fine-tuning. Our work thus represents a critical step toward scalable and verifiable chemical intelligence in LLMs.
@article{arxiv.2509.16543,
title = {ChemOrch: Empowering LLMs with Chemical Intelligence via Synthetic Instructions},
author = {Yue Huang and Zhengzhe Jiang and Xiaonan Luo and Kehan Guo and Haomin Zhuang and Yujun Zhou and Zhengqing Yuan and Xiaoqi Sun and Jules Schleinitz and Yanbo Wang and Shuhao Zhang and Mihir Surve and Nitesh V Chawla and Olaf Wiest and Xiangliang Zhang},
journal= {arXiv preprint arXiv:2509.16543},
year = {2025}
}