English
Related papers

Related papers: DeepTRACE: Auditing Deep Research AI Systems for T…

200 papers

The ability to research and synthesize knowledge is central to human expertise and progress. A new class of AI systems--designed for generative research synthesis--aims to automate this process by retrieving information from the live web…

Computation and Language · Computer Science 2026-02-10 Liana Patel , Negar Arabzadeh , Harshit Gupta , Ankita Sundar , Ion Stoica , Matei Zaharia , Carlos Guestrin

Large Language Models generate complex reasoning chains that reveal their decision-making, yet verifying the faithfulness and harmlessness of these intermediate steps remains a critical unsolved problem. Existing auditing methods are…

Artificial Intelligence · Computer Science 2025-10-24 Morris Yu-Chao Huang , Zhen Tan , Mohan Zhang , Pingzhi Li , Zhuo Zhang , Tianlong Chen

Achieving the right amount of trust in AI systems is important, but challenging. The problem is exacerbated with the rise of Large Language Models (LLMs) as they provide human-level communication capabilities, but potentially hallucinate in…

Information Retrieval · Computer Science 2026-05-05 Daan Di Scala , Maaike de Boer , Pınar Yolum

Large language models (LLMs) present a promising yet challenging frontier for automated source citation in scientific communication. Previous approaches to citation generation have been limited by citation ambiguity and LLM…

Computation and Language · Computer Science 2025-04-14 Yash Saxena , Deepa Tilwani , Ali Mohammadi , Edward Raff , Amit Sheth , Srinivasan Parthasarathy , Manas Gaur

In an era of AI-generated misinformation flooding the web, existing tools struggle to empower users with nuanced, transparent assessments of content credibility. They often default to binary (true/false) classifications without contextual…

Information Retrieval · Computer Science 2026-04-03 Joydeep Chandra , Aleksandr Algazinov , Satyam Kumar Navneet , Rim El Filali , Matt Laing , Andrew Hanna , Yong Zhang

This survey examines the rapidly evolving field of Deep Research systems -- AI-powered applications that automate complex research workflows through the integration of large language models, advanced information retrieval, and autonomous…

Artificial Intelligence · Computer Science 2025-06-17 Renjun Xu , Jingwen Peng

Retrieval-Augmented Generation (RAG) delivers substantial value in knowledge-intensive applications. However, its generated responses often lack transparent reasoning paths that trace back to source evidence from retrieved documents. This…

Computation and Language · Computer Science 2026-01-30 Jingyi Ren , Yekun Xu , Xiaolong Wang , Weitao Li , Ante Wang , Weizhi Ma , Yang Liu

Large Language Models (LLMs) excel at many reasoning tasks but struggle with knowledge-intensive queries due to their inability to dynamically access up-to-date or domain-specific information. Retrieval-Augmented Generation (RAG) has…

Computation and Language · Computer Science 2026-03-03 Minghao Guo , Qingcheng Zeng , Xujiang Zhao , Yanchi Liu , Wenchao Yu , Mengnan Du , Haifeng Chen , Wei Cheng

We study large-scale literature search from two complementary angles: improving the retrieval pipeline, and stress-testing the human reference list as an evaluation target. First, we implement a Deep Research pipeline that processes the…

Artificial Intelligence · Computer Science 2026-05-29 Gaurav Sahu , Laurent Charlin , Christopher Pal

Recent advances in retrieval-augmented generation (RAG) furnish large language models (LLMs) with iterative retrievals of relevant information to handle complex multi-hop questions. These methods typically alternate between LLM reasoning…

Computation and Language · Computer Science 2025-05-27 Rui Li , Quanyu Dai , Zeyu Zhang , Xu Chen , Zhenhua Dong , Ji-Rong Wen

Large language models (LLMs) increasingly answer queries by citing web sources, but existing evaluations emphasize answer correctness rather than evidence quality. We introduce SourceBench, a benchmark for measuring the quality of cited web…

Artificial Intelligence · Computer Science 2026-02-20 Hexi Jin , Stephen Liu , Yuheng Li , Simran Malik , Yiying Zhang

Large Language Models (LLMs) increasingly power generative search engines which, in turn, drive human information seeking and decision making at scale. The extent to which humans trust generative artificial intelligence (GenAI) can…

Computers and Society · Computer Science 2025-04-10 Haiwen Li , Sinan Aral

The evolution of Large Language Models (LLMs) towards autonomous agents has catalyzed progress in Deep Research. While retrieval capabilities are well-benchmarked, the post-retrieval synthesis stage--where agents must digest massive amounts…

Computation and Language · Computer Science 2026-01-08 Hongzhi Zhang , Yuanze Hu , Tinghai Zhang , Jia Fu , Tao Wang , Junwei Jing , Zhaoxin Fan , Qi Wang , Ruiming Tang , Han Li , Guorui Zhou , Kun Gai

Deep Research Agents increasingly automate survey generation, yet whether they match human experts at retrieving essential papers and organizing them into expert-like taxonomies remains unclear. Existing benchmarks emphasize writing quality…

Deep research systems are widely used for multi-step web research, analysis, and cross-source synthesis, yet their evaluation remains challenging. Existing benchmarks often require annotation-intensive task construction, rely on static…

Computation and Language · Computer Science 2026-01-15 Yibo Wang , Lei Wang , Yue Deng , Keming Wu , Yao Xiao , Huanjin Yao , Liwei Kang , Hai Ye , Yongcheng Jing , Lidong Bing

AI-powered answer engines are inherently non-deterministic: identical queries submitted at different times can produce different responses and cite different sources. Despite this stochastic behavior, current approaches to measuring domain…

Applications · Statistics 2026-03-11 Ronald Sielinski

Large language models (LLMs) are increasingly being used for complex research tasks such as literature review, idea generation, and scientific paper analysis, yet their ability to truly understand and process the intricate relationships…

Computation and Language · Computer Science 2025-06-11 Shashidhar Reddy Javaji , Yupeng Cao , Haohang Li , Yangyang Yu , Nikhil Muralidhar , Zining Zhu

Supervised fine-tuning with synthesized instructions has been a common practice for adapting LLMs to domain-specific QA tasks. However, the synthesized instructions deviate from real user questions and expected answers. This study proposes…

Computation and Language · Computer Science 2025-02-14 Yang Li , Mingxuan Luo , Yeyun Gong , Chen Lin , Jian Jiao , Yi Liu , Kaili Huang

Large Language Models (LLMs) like ChatGPT, DeepSeek and Gemini seem to be increasingly used for knowledge discovery, information retrieval, and knowledge summaries, including for academic topics. This can result in users being misled, such…

Information Retrieval · Computer Science 2026-05-28 Mike Thelwall

With increasing awareness of the hallucination risks of generative artificial intelligence (AI), we see a growing shift toward providing information tooling to help users determine the veracity of AI-generated answers for themselves. User…

Human-Computer Interaction · Computer Science 2026-03-13 Jessica Irons , Patrick Cooper , Necva Bolucu , Roelien Timmer , Huichen Yang , Changhyun Lee , Brian Jin , Andreas Duenser , Stephen Wan