English
Related papers

Related papers: DICE: Discrete Interpretable Comparative Evaluatio…

200 papers

Retrieval-Augmented Generation (RAG) is an effective approach to enhance the factual accuracy of large language models (LLMs) by retrieving information from external databases, which are typically composed of diverse sources, to supplement…

Machine Learning · Computer Science 2025-10-15 Jeongyeon Hwang , Junyoung Park , Hyejin Park , Dongwoo Kim , Sangdon Park , Jungseul Ok

Performance evaluation of Retrieval-Augmented Generation (RAG) systems within enterprise environments is governed by multi-dimensional and composite factors extending far beyond simple final accuracy checks. These factors include reasoning…

Computation and Language · Computer Science 2026-04-06 Kenichirou Narita , Siqi Peng , Taku Fukui , Moyuru Yamada , Satoshi Munakata , Satoru Takahashi

Retrieval-Augmented Generation (RAG) has emerged as a critical technique for enhancing large language model (LLM) capabilities. However, practitioners face significant challenges when making RAG deployment decisions. While existing research…

Software Engineering · Computer Science 2025-07-22 Shengming Zhao , Yuchen Shao , Yuheng Huang , Jiayang Song , Zhijie Wang , Chengcheng Wan , Lei Ma

Recently, building retrieval-augmented generation (RAG) systems to enhance the capability of large language models (LLMs) has become a common practice. Especially in the legal domain, previous judicial decisions play a significant role…

Computation and Language · Computer Science 2025-04-28 Minhu Park , Hongseok Oh , Eunkyung Choi , Wonseok Hwang

When performing reasoning tasks with user-specific requirements, such as strict output formats, large language models (LLMs) often prioritize reasoning over adherence to detailed instructions. Fine-tuning LLMs on supervised datasets to…

Computation and Language · Computer Science 2025-10-21 Yiqi Li , Yusheng Liao , Zhe Chen , Yanfeng Wang , Yu Wang

The score function estimator is widely used for estimating gradients of stochastic objectives in stochastic computation graphs (SCG), eg, in reinforcement learning and meta-learning. While deriving the first-order gradient estimators by…

Machine Learning · Computer Science 2018-11-07 Jakob Foerster , Gregory Farquhar , Maruan Al-Shedivat , Tim Rocktäschel , Eric P. Xing , Shimon Whiteson

While Retrieval Augmented Generation (RAG) is now widely adopted to enhance LLMs, evaluating its true performance benefits in a reproducible and interpretable way remains a major hurdle. Existing methods often fall short: they lack domain…

Information Retrieval · Computer Science 2025-08-11 Jiaxuan Liang , Shide Zhou , Kailong Wang

Retrieval-augmented generation (RAG) has become a powerful framework for enhancing large language models in knowledge-intensive and reasoning tasks. However, as reasoning chains deepen or search trees expand, RAG systems often face two…

Information Retrieval · Computer Science 2026-01-19 Shuguang Jiao , Xinyu Xiao , Yunfan Wei , Shuhan Qi , Chengkai Huang , Quan Z. Michael Sheng , Lina Yao

Retrieval-Augmented Generation (RAG) grounds large language model outputs in external evidence, but remains challenged on multi-hop question answering that requires long reasoning. Recent works scale RAG at inference time along two…

Retrieval-augmented generation (RAG) enhances language models by integrating external knowledge, but its effectiveness is highly dependent on system configuration. Improper retrieval settings can degrade performance, making RAG less…

Computation and Language · Computer Science 2025-07-17 Jennifer Hsia , Afreen Shaikh , Zhiruo Wang , Graham Neubig

Generative search engines and deep research LLM agents promise trustworthy, source-grounded synthesis, yet users regularly encounter overconfidence, weak sourcing, and confusing citation practices. We introduce DeepTRACE, a novel…

Computation and Language · Computer Science 2025-09-08 Pranav Narayanan Venkit , Philippe Laban , Yilun Zhou , Kung-Hsiang Huang , Yixin Mao , Chien-Sheng Wu

Retrieval-Augmented Generation (RAG) has gained significant attention in recent years for its potential to enhance natural language understanding and generation by combining large-scale retrieval systems with generative models. RAG…

Computation and Language · Computer Science 2025-03-18 Mingyue Cheng , Yucong Luo , Jie Ouyang , Qi Liu , Huijie Liu , Li Li , Shuo Yu , Bohou Zhang , Jiawei Cao , Jie Ma , Daoyu Wang , Enhong Chen

Retrieval-Augmented Generation (RAG) has become a standard architectural pattern for incorporating domain-specific knowledge into user-facing chat applications powered by Large Language Models (LLMs). RAG systems are characterized by (1) a…

Computation and Language · Computer Science 2025-01-17 Robert Friel , Masha Belyi , Atindriyo Sanyal

Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm to enhance large language models (LLMs) by conditioning generation on external evidence retrieved at inference time. While RAG addresses critical limitations of…

Information Retrieval · Computer Science 2025-06-03 Chaitanya Sharma

Test-time scaling has emerged as a transformative paradigm for enhancing the performance of large reasoning models, enabling dynamic allocation of computational resources during inference. However, as the landscape of reasoning models…

Artificial Intelligence · Computer Science 2025-10-08 Zhangyue Yin , Qiushi Sun , Zhiyuan Zeng , Zhiyuan Yu , Qipeng Guo , Xuanjing Huang , Xipeng Qiu

Retrieval-augmented generation (RAG) enhances large language models (LLMs) with external knowledge to answer questions more accurately. However, research on evaluating RAG systems-particularly the retriever component-remains limited, as…

Information Retrieval · Computer Science 2026-04-21 Lorenz Brehme , Thomas Ströhle , Ruth Breu

Language models (LMs) are increasingly being integrated into a wide range of applications, yet the modern evaluation paradigm does not sufficiently reflect how they are actually being used. Current evaluations rely on benchmarks that often…

Computation and Language · Computer Science 2025-04-15 Aryan Shrivastava , Paula Akemi Aoyagui

This paper introduces a novel method for closed information extraction. The method employs a discriminative approach that incorporates type and entity-specific information to improve relation extraction accuracy, particularly benefiting…

Computation and Language · Computer Science 2025-06-23 Cedric Möller , Ricardo Usbeck

Stationary Distribution Correction Estimation (DICE) addresses the mismatch between the stationary distribution induced by a policy and the target distribution required for reliable off-policy evaluation (OPE) and policy optimization.…

Machine Learning · Computer Science 2025-06-11 Woosung Kim , JunHo Seo , Jongmin Lee , Byung-Jun Lee

Agentic Retrieval-Augmented Generation (Agentic RAG) enhances the processing capability for complex tasks through dynamic retrieval and adaptive workflows. Recent advances (e.g., Search-R1) have shown that outcome-supervised reinforcement…

Computation and Language · Computer Science 2025-10-08 Yongqi Leng , Yikun Lei , Xikai Liu , Meizhi Zhong , Bojian Xiong , Yurong Zhang , Yan Gao , Yi Wu , Yao Hu , Deyi Xiong