PharmaGPT: Domain-Specific Large Language Models for Bio-Pharmaceutical and Chemistry
Abstract
Large language models (LLMs) have revolutionized Natural Language Processing (NLP) by minimizing the need for complex feature engineering. However, the application of LLMs in specialized domains like biopharmaceuticals and chemistry remains largely unexplored. These fields are characterized by intricate terminologies, specialized knowledge, and a high demand for precision areas where general purpose LLMs often fall short. In this study, we introduce PharmaGPT, a suite of domain specilized LLMs with 13 billion and 70 billion parameters, specifically trained on a comprehensive corpus tailored to the Bio-Pharmaceutical and Chemical domains. Our evaluation shows that PharmaGPT surpasses existing general models on specific-domain benchmarks such as NAPLEX, demonstrating its exceptional capability in domain-specific tasks. Remarkably, this performance is achieved with a model that has only a fraction, sometimes just one-tenth-of the parameters of general-purpose large models. This advancement establishes a new benchmark for LLMs in the bio-pharmaceutical and chemical fields, addressing the existing gap in specialized language modeling. It also suggests a promising path for enhanced research and development, paving the way for more precise and effective NLP applications in these areas.
Cite
@article{arxiv.2406.18045,
title = {PharmaGPT: Domain-Specific Large Language Models for Bio-Pharmaceutical and Chemistry},
author = {Linqing Chen and Weilei Wang and Zilong Bai and Peng Xu and Yan Fang and Jie Fang and Wentao Wu and Lizhi Zhou and Ruiji Zhang and Yubin Xia and Chaobo Xu and Ran Hu and Licong Xu and Qijun Cai and Haoran Hua and Jing Sun and Jin Liu and Tian Qiu and Haowen Liu and Meng Hu and Xiuwen Li and Fei Gao and Yufu Wang and Lin Tie and Chaochao Wang and Jianping Lu and Cheng Sun and Yixin Wang and Shengjie Yang and Yuancheng Li and Lu Jin and Lisha Zhang and Fu Bian and Zhongkai Ye and Lidong Pei and Changyang Tu},
journal= {arXiv preprint arXiv:2406.18045},
year = {2024}
}