English
Related papers

Related papers: Bridging Text and Molecule: A Survey on Multimodal…

200 papers

Discovering new medicines is the hallmark of human endeavor to live a better and longer life. Yet the pace of discovery has slowed down as we need to venture into more wildly unexplored biomedical space to find one that matches today's high…

Artificial Intelligence · Computer Science 2022-02-16 Tri Minh Nguyen , Thin Nguyen , Truyen Tran

Human expertise in chemistry and biomedicine relies on contextual molecular understanding, a capability that large language models (LLMs) can extend through fine-grained alignment between molecular structures and text. Recent multimodal…

Computation and Language · Computer Science 2025-03-10 Sumin Ha , Jun Hyeong Kim , Yinhua Piao , Sun Kim

Understanding molecules is key to understanding organisms and driving advances in drug discovery, requiring interdisciplinary knowledge across chemistry and biology. Although large molecular language models have achieved notable success in…

Machine Learning · Computer Science 2025-10-03 Dongki Kim , Wonbin Lee , Sung Ju Hwang

In many scientific fields, large language models (LLMs) have revolutionized the way text and other modalities of data (e.g., molecules and proteins) are handled, achieving superior performance in various applications and augmenting the…

Computation and Language · Computer Science 2024-10-01 Yu Zhang , Xiusi Chen , Bowen Jin , Sheng Wang , Shuiwang Ji , Wei Wang , Jiawei Han

This paper provides a comprehensive survey of the latest research on multilingual large language models (MLLMs). MLLMs not only are able to understand and generate language across linguistic boundaries, but also represent an important…

Computation and Language · Computer Science 2024-11-20 Shaolin Zhu , Supryadi , Shaoyang Xu , Haoran Sun , Leiyu Pan , Menglong Cui , Jiangcun Du , Renren Jin , António Branco , Deyi Xiong

The quest for accurate prediction of drug molecule properties poses a fundamental challenge in the realm of Artificial Intelligence Drug Discovery (AIDD). An effective representation of drug molecules emerges as a pivotal component in this…

Machine Learning · Computer Science 2024-04-22 Zhuoyuan Wang , Jiacong Mi , Shan Lu , Jieyue He

In the molecular domain, numerous studies have explored the use of multimodal large language models (LLMs) to construct a general-purpose, multi-task molecular model. However, these efforts are still far from achieving a truly universal…

Machine Learning · Computer Science 2025-10-31 Chengxin Hu , Hao Li , Yihe Yuan , Zezheng Song , Chenyang Zhao , Haixin Wang

Large language models have made significant strides in natural language processing, enabling innovative applications in molecular science by processing textual representations of molecules. However, most existing language models cannot…

Machine Learning · Computer Science 2024-02-07 Pengfei Liu , Yiming Ren , Jun Tao , Zhixiang Ren

Artificial Intelligence (AI) and Machine Learning (ML) are weaving their way into the fabric of society, where they are playing a crucial role in numerous facets of our lives. As we witness the increased deployment of AI and ML in various…

Emerging Technologies · Computer Science 2022-12-23 Sasitharan Balasubramaniam , Samitha Somathilaka , Sehee Sun , Adrian Ratwatte , Massimiliano Pierobon

Deep Learning and its applications have cascaded impactful research and development with a diverse range of modalities present in the real-world data. More recently, this has enhanced research interests in the intersection of the Vision and…

Computer Vision and Pattern Recognition · Computer Science 2020-12-23 Shagun Uppal , Sarthak Bhagat , Devamanyu Hazarika , Navonil Majumdar , Soujanya Poria , Roger Zimmermann , Amir Zadeh

Multimodal large language models (MLLMs) enhance the capabilities of standard large language models by integrating and processing data from multiple modalities, including text, vision, audio, video, and 3D environments. Data plays a pivotal…

Artificial Intelligence · Computer Science 2024-07-19 Tianyi Bai , Hao Liang , Binwang Wan , Yanran Xu , Xi Li , Shiyu Li , Ling Yang , Bozhou Li , Yifan Wang , Bin Cui , Ping Huang , Jiulong Shan , Conghui He , Binhang Yuan , Wentao Zhang

In this review, we highlight recent developments in the application of machine learning for molecular modeling and simulation. After giving a brief overview of the foundations, components, and workflow of a typical supervised learning…

Data Analysis, Statistics and Probability · Physics 2019-02-21 Mojtaba Haghighatlari , Johannes Hachmann

Multimodal Machine Learning (MML) aims to integrate and analyze information from diverse modalities, such as text, audio, and visuals, enabling machines to address complex tasks like sentiment analysis, emotion recognition, and multimedia…

Computation and Language · Computer Science 2025-08-22 Abdelhamid Haouhat , Slimane Bellaouar , Attia Nehar , Hadda Cherroun , Ahmed Abdelali

Recent advancements in biology and chemistry have leveraged multi-modal learning, integrating molecules and their natural language descriptions to enhance drug discovery. However, current pre-training frameworks are limited to two…

Machine Learning · Computer Science 2025-02-05 Teng Xiao , Chao Cui , Huaisheng Zhu , Vasant G. Honavar

Molecular knowledge resides within three different modalities of information sources: molecular structures, biomedical documents, and knowledge bases. Effective incorporation of molecular knowledge from these modalities holds paramount…

Biomolecules · Quantitative Biology 2023-07-24 Yizhen Luo , Kai Yang , Massimo Hong , Xing Yi Liu , Zaiqing Nie

In the real world, a molecule is a 3D geometric structure. Compared to 1D SMILES sequences and 2D molecular graphs, 3D molecules represent the most informative molecular modality. Despite the rapid progress of autoregressive-based language…

Computational Engineering, Finance, and Science · Computer Science 2025-08-15 Lei Jiang , Shuzhou Sun , Biqing Qi , Yuchen Fu , Xiaohua Xu , Yuqiang Li , Dongzhan Zhou , Tianfan Fu

Our experience of the world is multimodal - we see objects, hear sounds, feel texture, smell odors, and taste flavors. Modality refers to the way in which something happens or is experienced and a research problem is characterized as…

Machine Learning · Computer Science 2017-08-02 Tadas Baltrušaitis , Chaitanya Ahuja , Louis-Philippe Morency

We present a perspective on molecular machine learning (ML) in the field of chemical process engineering. Recently, molecular ML has demonstrated great potential in (i) providing highly accurate predictions for properties of pure components…

Chemical Physics · Physics 2025-09-01 Jan G. Rittig , Manuel Dahmen , Martin Grohe , Philippe Schwaller , Alexander Mitsos

Understanding molecular structure and related knowledge is crucial for scientific research. Recent studies integrate molecular graphs with their textual descriptions to enhance molecular representation learning. However, they focus on the…

Artificial Intelligence · Computer Science 2025-06-02 Yibo Li , Yuan Fang , Mengmei Zhang , Chuan Shi

Large Language Models (LLMs) stand at the forefront of a number of Natural Language Processing (NLP) tasks. Despite the widespread adoption of LLMs in NLP, much of their potential in broader fields remains largely unexplored, and…

Machine Learning · Computer Science 2024-03-11 Zhiqiang Zhong , Kuangyu Zhou , Davide Mottin