English
Related papers

Related papers: Concept-Enhanced Multimodal RAG: Towards Interpret…

200 papers

Retrieval-Augmented Generation (RAG) systems have revolutionized information retrieval and question answering, but traditional text-based chunking methods struggle with complex document structures, multi-page tables, embedded figures, and…

Machine Learning · Computer Science 2025-07-15 Vishesh Tripathi , Tanmay Odapally , Indraneel Das , Uday Allu , Biddwan Ahmed

Automated Radiology report generation (RRG) aims at producing detailed descriptions of medical images, reducing radiologists' workload and improving access to high-quality diagnostic services. Existing encoder-decoder models only rely on…

Computer Vision and Pattern Recognition · Computer Science 2025-05-13 Quang Vinh Nguyen , Minh Duc Nguyen , Thanh Hoang Son Vo , Hyung-Jeong Yang , Soo-Hyung Kim

Multimodal Retrieval-Augmented Generation (RAG) has emerged as an effective paradigm for enhancing Large Language Models (LLMs) with external knowledge. However, existing multimodal RAG systems predominantly rely on coarse-grained…

Information Retrieval · Computer Science 2026-05-25 Yifan Zhu , Yu Mi , Yue Lu , Yanchu Guan , Zhixuan Chu

The objective of Radiology Report Generation (RRG) is to automatically generate coherent textual analyses of diseases based on radiological images, thereby alleviating the workload of radiologists. Current AI-based methods for RRG primarily…

Computer Vision and Pattern Recognition · Computer Science 2024-11-21 Tiancheng Gu , Kaicheng Yang , Xiang An , Ziyong Feng , Dongnan Liu , Weidong Cai

Clinical decision-making in radiology increasingly benefits from artificial intelligence (AI), particularly through large language models (LLMs). However, traditional retrieval-augmented generation (RAG) systems for radiology question…

Multimodal Retrieval-Augmented Generation (MRAG) enhances reasoning capabilities by integrating external knowledge. However, existing benchmarks primarily focus on simple image-text interactions, overlooking complex visual formats like…

Artificial Intelligence · Computer Science 2025-02-21 Yuming Yang , Jiang Zhong , Li Jin , Jingwang Huang , Jingpeng Gao , Qing Liu , Yang Bai , Jingyuan Zhang , Rui Jiang , Kaiwen Wei

Understanding information from a collection of multiple documents, particularly those with visually rich elements, is important for document-grounded question answering. This paper introduces VisDoMBench, the first comprehensive benchmark…

Computation and Language · Computer Science 2025-02-12 Manan Suri , Puneet Mathur , Franck Dernoncourt , Kanika Goswami , Ryan A. Rossi , Dinesh Manocha

With the rapid advancement of Multi-modal Large Language Models (MLLMs), their capability in understanding both images and text has greatly improved. However, their potential for leveraging multi-modal contextual information in…

Artificial Intelligence · Computer Science 2025-08-08 Zhenghao Liu , Xingsheng Zhu , Tianshuo Zhou , Xinyi Zhang , Xiaoyuan Yi , Yukun Yan , Ge Yu , Maosong Sun

Large language models (LLMs) are transforming the landscape of medicine, yet two fundamental challenges persist: keeping up with rapidly evolving medical knowledge and providing verifiable, evidence-grounded reasoning. Retrieval-augmented…

Medical image segmentation is crucial for clinical diagnosis, yet existing models are limited by their reliance on explicit human instructions and lack the active reasoning capabilities to understand complex clinical questions. While recent…

Computer Vision and Pattern Recognition · Computer Science 2025-06-13 Yu Huang , Zelin Peng , Yichen Zhao , Piao Yang , Xiaokang Yang , Wei Shen

Automated radiology report drafting (ARRD) using vision-language models (VLMs) has advanced rapidly, yet most systems lack explicit uncertainty estimates, limiting trust and safe clinical deployment. We propose CONRep, a model-agnostic…

Large language models (LLMs), including zero-shot and few-shot paradigms, have shown promising capabilities in clinical text generation. However, real-world applications face two key challenges: (1) patient data is highly unstructured,…

Computation and Language · Computer Science 2025-07-10 Garapati Keerthana , Manik Gupta

Sleep stage classification based on electroencephalography (EEG) is fundamental for assessing sleep quality and diagnosing sleep-related disorders. However, most traditional machine learning methods rely heavily on prior knowledge and…

Artificial Intelligence · Computer Science 2025-11-25 Xihe Qiu , Gengchen Ma , Haoyu Wang , Chen Zhan , Xiaoyu Tan , Shuo Li

Large language models (LLMs) in biomedicine face a fundamental conflict between static parameter knowledge and the dynamic nature of clinical evidence. Retrieval-Augmented Generation (RAG) addresses this by grounding generation in external…

Other Quantitative Biology · Quantitative Biology 2025-12-19 Jiawei He , Boya Zhang , Hossein Rouhizadeh , Yingjian Chen , Rui Yang , Jin Lu , Xudong Chen , Nan Liu , Douglas Teodoro

The rapid evolution of Retrieval-Augmented Generation (RAG) toward multimodal, high-stakes enterprise applications has outpaced the development of domain specific evaluation benchmarks. Existing datasets often rely on general-domain corpora…

Artificial Intelligence · Computer Science 2026-01-23 Chandan Kumar Sahu , Premith Kumar Chilukuri , Matthew Hetrich

Financial documents--such as 10-Ks, 10-Qs, and investor presentations--span hundreds of pages and combine diverse modalities, including dense narrative text, structured tables, and complex figures. Answering questions over such content…

Computation and Language · Computer Science 2026-04-13 Chinmay Gondhalekar , Urjitkumar Patel , Fang-Chun Yeh

Deep learning based automated pathological diagnosis has markedly improved diagnostic efficiency and reduced variability between observers, yet its clinical adoption remains limited by opaque model decisions and a lack of traceable…

Large Language Models (LLMs), although powerful in general domains, often perform poorly on domain-specific tasks such as medical question answering (QA). In addition, LLMs tend to function as "black-boxes", making it challenging to modify…

Computation and Language · Computer Science 2024-08-19 Yucheng Shi , Shaochen Xu , Tianze Yang , Zhengliang Liu , Tianming Liu , Quanzheng Li , Xiang Li , Ninghao Liu

Question answering over visually rich documents (VRDs) requires reasoning not only over isolated content but also over documents' structural organization and cross-page dependencies. However, conventional retrieval-augmented generation…

Computation and Language · Computer Science 2026-03-03 Zhivar Sourati , Zheng Wang , Marianne Menglin Liu , Yazhe Hu , Mengqing Guo , Sujeeth Bharadwaj , Kyu Han , Tao Sheng , Sujith Ravi , Morteza Dehghani , Dan Roth

Retrieval-Augmented Generation (RAG) has been shown to enhance the factual accuracy of Large Language Models (LLMs), but existing methods often suffer from limited reasoning capabilities in effectively using the retrieved evidence,…

Computation and Language · Computer Science 2024-10-03 Shayekh Bin Islam , Md Asib Rahman , K S M Tozammel Hossain , Enamul Hoque , Shafiq Joty , Md Rizwan Parvez