English
Related papers

Related papers: When RAG Hurts: Diagnosing and Mitigating Attentio…

200 papers

Retrieval Augmented Generation (RAG) has emerged as a crucial technique for enhancing the accuracy of Large Language Models (LLMs) by incorporating external information. With the advent of LLMs that support increasingly longer context…

Machine Learning · Computer Science 2024-11-07 Quinn Leng , Jacob Portes , Sam Havens , Matei Zaharia , Michael Carbin

Retrieval-Augmented Generation (RAG) systems have recently shown remarkable advancements by integrating retrieval mechanisms into language models, enhancing their ability to produce more accurate and contextually relevant responses.…

Computation and Language · Computer Science 2025-01-14 Siran Li , Linus Stenzel , Carsten Eickhoff , Seyed Ali Bahrainian

Large language models (LLMs) are transforming the landscape of medicine, yet two fundamental challenges persist: keeping up with rapidly evolving medical knowledge and providing verifiable, evidence-grounded reasoning. Retrieval-augmented…

With the rapid advancement of Multi-modal Large Language Models (MLLMs), their capability in understanding both images and text has greatly improved. However, their potential for leveraging multi-modal contextual information in…

Artificial Intelligence · Computer Science 2025-08-08 Zhenghao Liu , Xingsheng Zhu , Tianshuo Zhou , Xinyi Zhang , Xiaoyuan Yi , Yukun Yan , Ge Yu , Maosong Sun

Retrieval-Augmented Generation (RAG) has emerged as a prominent method for incorporating domain knowledge into Large Language Models (LLMs). While RAG enhances response relevance by incorporating retrieved domain knowledge in the context,…

Computation and Language · Computer Science 2025-03-28 Kushagra Bhushan , Yatin Nandwani , Dinesh Khandelwal , Sonam Gupta , Gaurav Pandey , Dinesh Raghu , Sachindra Joshi

Artificial Intelligence (AI) has demonstrated significant potential in healthcare, particularly in disease diagnosis and treatment planning. Recent progress in Medical Large Vision-Language Models (Med-LVLMs) has opened up new possibilities…

Machine Learning · Computer Science 2025-03-04 Peng Xia , Kangyu Zhu , Haoran Li , Tianze Wang , Weijia Shi , Sheng Wang , Linjun Zhang , James Zou , Huaxiu Yao

A well-known issue with Retrieval Augmented Generation (RAG) is that retrieved passages that are irrelevant to the query sometimes distract the answer-generating LLM, causing it to provide an incorrect response. In this paper, we shed light…

Computation and Language · Computer Science 2025-10-29 Chen Amiraz , Florin Cuconasu , Simone Filice , Zohar Karnin

Retrieval-Augmented Generation (RAG) has emerged as the predominant paradigm for grounding Large Language Model outputs in factual knowledge, effectively mitigating hallucinations. However, conventional RAG systems operate under a…

Information Retrieval · Computer Science 2026-01-13 Sergii Voloshyn

Retrieval-Augmented Generation (RAG) has become a key paradigm for reducing factual hallucinations in Large Language Models (LLMs), yet little is known about how the order of retrieved documents affects model behavior. We empirically show…

Computation and Language · Computer Science 2026-04-22 Qianchi Zhang , Hainan Zhang , Liang Pang , Hongwei Zheng , Zhiming Zheng

Question answering over visually rich documents (VRDs) requires reasoning not only over isolated content but also over documents' structural organization and cross-page dependencies. However, conventional retrieval-augmented generation…

Computation and Language · Computer Science 2026-03-03 Zhivar Sourati , Zheng Wang , Marianne Menglin Liu , Yazhe Hu , Mengqing Guo , Sujeeth Bharadwaj , Kyu Han , Tao Sheng , Sujith Ravi , Morteza Dehghani , Dan Roth

Retrieval-augmented generation (RAG) with large language models (LLMs) has demonstrated strong performance in multilingual question-answering (QA) tasks by leveraging relevant passages retrieved from corpora. In multilingual RAG (mRAG), the…

Computation and Language · Computer Science 2025-12-12 Jirui Qi , Raquel Fernández , Arianna Bisazza

Retrieval-Augmented Generation (RAG) aims to reduce hallucination by grounding answers in retrieved evidence, yet hallucinated answers remain common even when relevant documents are available. Existing evaluations focus on answer-level or…

Computation and Language · Computer Science 2026-05-21 Passant Elchafei , Monorama Swain , Shahed Masoudian , Markus Schedl

Despite the remarkable capabilities of Large Vision Language Models (LVLMs), they still lack detailed knowledge about specific entities. Retrieval-augmented Generation (RAG) is a widely adopted solution that enhances LVLMs by providing…

Computation and Language · Computer Science 2026-02-09 Jongha Kim , Byungoh Ko , Jeehye Na , Jinsung Yoon , Hyunwoo J. Kim

Retrieval-Augmented Generation (RAG) has recently demonstrated the performance of Large Language Models (LLMs) in the knowledge-intensive tasks such as Question-Answering (QA). RAG expands the query context by incorporating external…

Machine Learning · Computer Science 2024-06-18 Zijian Hei , Weiling Liu , Wenjie Ou , Juyi Qiao , Junming Jiao , Guowen Song , Ting Tian , Yi Lin

Retrieval Augmented Generation (RAG) has emerged as a widely adopted approach to mitigate the limitations of large language models (LLMs) in answering domain-specific questions. Previous research has predominantly focused on improving the…

Machine Learning · Computer Science 2025-01-07 Mohammad Hassan Heydari , Arshia Hemmat , Erfan Naman , Afsaneh Fatemi

Retrieval-Augmented Generation (RAG) has become a widely adopted paradigm for enhancing the reliability of large language models (LLMs). However, RAG systems are sensitive to retrieval strategies that rely on text chunking to construct…

Information Retrieval · Computer Science 2026-03-31 Sun Xu , Tongkai Xu , Baiheng Xie , Li Huang , Qiang Gao , Kunpeng Zhang

Large Language Models (LLMs) are smart but forgetful. Recent studies, (e.g., (Bubeck et al., 2023)) on modern LLMs have shown that they are capable of performing amazing tasks typically necessitating human-level intelligence. However,…

Computation and Language · Computer Science 2023-11-08 Eric Melz

Recent advancements in Large language models (LLMs) have demonstrated remarkable capabilities across diverse domains. While they exhibit strong zero-shot performance on various tasks, LLMs' effectiveness in music-related applications…

Computation and Language · Computer Science 2025-12-09 Daeyong Kwon , SeungHeon Doh , Juhan Nam

Retrieval-Augmented Generation (RAG) has been shown to enhance the factual accuracy of Large Language Models (LLMs), but existing methods often suffer from limited reasoning capabilities in effectively using the retrieved evidence,…

Computation and Language · Computer Science 2024-10-03 Shayekh Bin Islam , Md Asib Rahman , K S M Tozammel Hossain , Enamul Hoque , Shafiq Joty , Md Rizwan Parvez

Retrieval-Augmented Generation (RAG) has emerged as a powerful framework to improve factuality in large language models (LLMs) by grounding their outputs in retrieved documents. However, ensuring perfect retrieval of relevant information…

Computation and Language · Computer Science 2025-12-04 Zhan Peng Lee , Andre Lin , Calvin Tan