中文
相关论文

相关论文: Multi-modal Retrieval Augmented Multi-modal Genera…

200 篇论文

Current Retrieval-Augmented Generation (RAG) systems primarily operate on unimodal textual data, limiting their effectiveness on unstructured multimodal documents. Such documents often combine text, images, tables, equations, and graphs,…

机器学习 · 计算机科学 2025-10-17 Rashmi R , Vidyadhar Upadhya

Retrieval-Augmented Generation (RAG) has recently been extended to multimodal settings, connecting multimodal large language models (MLLMs) with vast corpora of external knowledge such as multimodal knowledge graphs (MMKGs). Despite their…

计算与语言 · 计算机科学 2026-04-14 Hyeongcheol Park , Jiyoung Seo , Jaewon Mun , Hogun Park , Wonmin Byeon , Sung June Kim , Hyeonsoo Im , JeungSub Lee , Sangpil Kim

Document understanding is critical for applications from financial analysis to scientific discovery. Current approaches, whether OCR-based pipelines feeding Large Language Models (LLMs) or native Multimodal LLMs (MLLMs), face key…

Retrieval-augmented generation (RAG) enables large language models (LLMs) to dynamically access external information, which is powerful for answering questions over previously unseen documents. Nonetheless, they struggle with high-level…

人工智能 · 计算机科学 2026-04-21 Chi-Hsiang Hsiao , Yi-Cheng Wang , Tzung-Sheng Lin , Yi-Ren Yeh , Chu-Song Chen

To effectively engage in human society, the ability to adapt, filter information, and make informed decisions in ever-changing situations is critical. As robots and intelligent agents become more integrated into human life, there is a…

Large language models equipped with retrieval-augmented generation (RAG) represent a burgeoning field aimed at enhancing answering capabilities by leveraging external knowledge bases. Although the application of RAG with language-only…

计算机视觉与模式识别 · 计算机科学 2024-06-03 Cheng Tan , Jingxuan Wei , Linzhuang Sun , Zhangyang Gao , Siyuan Li , Bihui Yu , Ruifeng Guo , Stan Z. Li

Retrieval Augmented Generation (RAG) has proven to be highly effective in boosting the generative performance of language model in knowledge-intensive tasks. However, existing RAG framework either indiscriminately perform retrieval or rely…

人工智能 · 计算机科学 2025-01-03 Xiaqiang Tang , Qiang Gao , Jian Li , Nan Du , Qi Li , Sihong Xie

With the rapid development of large-scale language models, Retrieval-Augmented Generation (RAG) has been widely adopted. However, existing RAG paradigms are inevitably influenced by erroneous retrieval information, thereby reducing the…

信息检索 · 计算机科学 2024-05-30 Ridong Wu , Shuhong Chen , Xiangbiao Su , Yuankai Zhu , Yifei Liao , Jianming Wu

Retrieval-augmented generation (RAG) has recently emerged as a promising solution for incorporating up-to-date or domain-specific knowledge into large language models (LLMs) and improving LLM factuality, but is predominantly studied in…

计算与语言 · 计算机科学 2024-07-02 Nadezhda Chirkova , David Rau , Hervé Déjean , Thibault Formal , Stéphane Clinchant , Vassilina Nikoulina

Multimodal Retrieval-Augmented Generation (MRAG) enhances reasoning capabilities by integrating external knowledge. However, existing benchmarks primarily focus on simple image-text interactions, overlooking complex visual formats like…

人工智能 · 计算机科学 2025-02-21 Yuming Yang , Jiang Zhong , Li Jin , Jingwang Huang , Jingpeng Gao , Qing Liu , Yang Bai , Jingyuan Zhang , Rui Jiang , Kaiwen Wei

Retrieval-augmented generation (RAG) improves large language models (LLMs) by using external knowledge to guide response generation, reducing hallucinations. However, RAG, particularly multi-modal RAG, can introduce new hallucination…

机器学习 · 计算机科学 2025-01-08 Matin Mortaheb , Mohammad A. Amir Khojastepour , Srimat T. Chakradhar , Sennur Ulukus

A Comparison of Independent and Joint Fine-tuning Strategies for Retrieval-Augmented Generation Download PDF Neal Gregory Lawton, Alfy Samuel, Anoop Kumar, Daben Liu Published: 20 Aug 2025, Retrieval augmented generation (RAG) is a popular…

计算与语言 · 计算机科学 2025-10-21 Neal Gregory Lawton , Alfy Samuel , Anoop Kumar , Daben Liu

We introduce a novel graph-based Retrieval-Augmented Generation (RAG) framework specifically designed for the medical domain, called \textbf{MedGraphRAG}, aimed at enhancing Large Language Model (LLM) capabilities for generating…

计算机视觉与模式识别 · 计算机科学 2024-10-17 Junde Wu , Jiayuan Zhu , Yunli Qi , Jingkun Chen , Min Xu , Filippo Menolascina , Vicente Grau

The multi-modal long-context document question-answering task aims to locate and integrate multi-modal evidences (such as texts, tables, charts, images, and layouts) distributed across multiple pages, for question understanding and answer…

多媒体 · 计算机科学 2025-10-06 Ziyu Gong , Chengcheng Mai , Yihua Huang

Retrieval-Augmented Generation (RAG) improves Large Language Models (LLMs) by retrieving supporting documents into the prompt, but existing methods do not explicitly target queries that require fetching multiple documents with substantially…

While Retrieval Augmented Generation (RAG) is now widely adopted to enhance LLMs, evaluating its true performance benefits in a reproducible and interpretable way remains a major hurdle. Existing methods often fall short: they lack domain…

信息检索 · 计算机科学 2025-08-11 Jiaxuan Liang , Shide Zhou , Kailong Wang

We introduce a novel retrieval-augmented generation (RAG) framework tailored for multihop question answering. First, our system uses large language model (LLM) to decompose complex multihop questions into a sequence of single-hop…

计算与语言 · 计算机科学 2025-08-14 Seokgi Lee

While language Models store a massive amount of world knowledge implicitly in their parameters, even very large models often fail to encode information about rare entities and events, while incurring huge computational costs. Recently,…

计算与语言 · 计算机科学 2022-10-21 Wenhu Chen , Hexiang Hu , Xi Chen , Pat Verga , William W. Cohen

Large language models (LLM) hold significant potential for applications in biomedicine, but they struggle with hallucinations and outdated knowledge. While retrieval-augmented generation (RAG) is generally employed to address these issues,…

计算与语言 · 计算机科学 2025-09-23 Jiwoong Sohn , Yein Park , Chanwoong Yoon , Sihyeon Park , Hyeon Hwang , Mujeen Sung , Hyunjae Kim , Jaewoo Kang

Developing the capacity to effectively search for requisite datasets is an urgent requirement to assist data users in identifying relevant datasets considering the very limited available metadata. For this challenge, the utilization of…

信息检索 · 计算机科学 2024-10-08 Teruaki Hayashi , Hiroki Sakaji , Jiayi Dai , Randy Goebel