English
Related papers

Related papers: Retrieval Models Aren't Tool-Savvy: Benchmarking T…

200 papers

Large Language Models (LLMs) have demonstrated remarkable zero-shot generalization across various language-related tasks, including search engines. However, existing work utilizes the generative ability of LLMs for Information Retrieval…

Computation and Language · Computer Science 2024-12-31 Weiwei Sun , Lingyong Yan , Xinyu Ma , Shuaiqiang Wang , Pengjie Ren , Zhumin Chen , Dawei Yin , Zhaochun Ren

Large Language Models (LLMs) often struggle with hallucinations and outdated information. To address this, Information Retrieval (IR) systems can be employed to augment LLMs with up-to-date knowledge. However, existing IR techniques contain…

Computation and Language · Computer Science 2024-11-26 Danupat Khamnuansin , Tawunrat Chalothorn , Ekapol Chuangsuwanich

Large language model (LLM) agents rely on external tools to solve complex tasks, but real-world toolsets often contain redundant tools with overlapping names and descriptions, introducing ambiguity and reducing selection accuracy. LLMs also…

Computation and Language · Computer Science 2026-05-12 Marianne Menglin Liu , Daniel Garcia , Fjona Parllaku , Vikas Upadhyay , Syed Fahad Allam Shah , Dan Roth

This paper addresses the gap between general-purpose text embeddings and the specific demands of item retrieval tasks. We demonstrate the shortcomings of existing models in capturing the nuances necessary for zero-shot performance on item…

Information Retrieval · Computer Science 2024-03-01 Yuxuan Lei , Jianxun Lian , Jing Yao , Mingqi Wu , Defu Lian , Xing Xie

Recent advances in large language models (LLMs) have enabled autonomous agents with complex reasoning and task-fulfillment capabilities using a wide range of tools. However, effectively identifying the most relevant tools for a given task…

Large Language Models (LLMs) ) have demonstrated promise in boosting productivity across AI-powered tools, yet existing benchmarks like Massive Multitask Language Understanding (MMLU) inadequately assess enterprise-specific task…

Artificial Intelligence · Computer Science 2025-06-26 Liya Wang , David Yi , Damien Jose , John Passarelli , James Gao , Jordan Leventis , Kang Li

Current Large Language Models (LLMs) often undergo supervised fine-tuning (SFT) to acquire tool use capabilities. However, SFT struggles to generalize to unfamiliar or complex tool use scenarios. Recent advancements in reinforcement…

Machine Learning · Computer Science 2025-04-22 Cheng Qian , Emre Can Acikgoz , Qi He , Hongru Wang , Xiusi Chen , Dilek Hakkani-Tür , Gokhan Tur , Heng Ji

Despite the advancements of open-source large language models (LLMs), e.g., LLaMA, they remain significantly limited in tool-use capabilities, i.e., using external tools (APIs) to fulfill human instructions. The reason is that current…

The development of autonomous machine learning (ML) agents capable of end-to-end data science workflows represents a significant frontier in artificial intelligence. These agents must orchestrate complex sequences of data analysis, feature…

Machine Learning · Computer Science 2026-02-24 Yaswanth Chittepu , Raghavendra Addanki , Tung Mai , Anup Rao , Branislav Kveton

Solving problems through tool use under explicit constraints constitutes a highly challenging yet unavoidable scenario for large language models (LLMs), requiring capabilities such as function calling, instruction following, and…

Computation and Language · Computer Science 2026-03-17 Junjie Ye , Guoqiang Zhang , Wenjie Fu , Tao Gui , Qi Zhang , Xuanjing Huang

As a primary means of information acquisition, information retrieval (IR) systems, such as search engines, have integrated themselves into our daily lives. These systems also serve as components of dialogue, question-answering, and…

Computation and Language · Computer Science 2025-09-18 Yutao Zhu , Huaying Yuan , Shuting Wang , Jiongnan Liu , Wenhan Liu , Chenlong Deng , Haonan Chen , Zheng Liu , Zhicheng Dou , Ji-Rong Wen

Recently, the fast development of Large Language Models (LLMs) such as ChatGPT has significantly advanced NLP tasks by enhancing the capabilities of conversational models. However, the application of LLMs in the recommendation domain has…

Information Retrieval · Computer Science 2023-08-24 Junling Liu , Chao Liu , Peilin Zhou , Qichen Ye , Dading Chong , Kang Zhou , Yueqi Xie , Yuwei Cao , Shoujin Wang , Chenyu You , Philip S. Yu

Large language models (LLMs) show an innate skill for solving language based tasks. But insights have suggested an inability to adjust for information or task-solving skills becoming outdated, as their knowledge, stored directly within…

Computation and Language · Computer Science 2024-04-16 Jerry Huang , Prasanna Parthasarathi , Mehdi Rezagholizadeh , Sarath Chandar

Tool-augmented large language models (LLMs) are often trained on datasets of query-response pairs, which embed the ability to use tools or APIs directly into the parametric knowledge of LLMs. Tool-augmented LLMs need the ability to forget…

Machine Learning · Computer Science 2025-08-07 Jiali Cheng , Hadi Amiri

Multi-stage information retrieval (IR) has become a widely-adopted paradigm in search. While Large Language Models (LLMs) have been extensively evaluated as second-stage reranking models for monolingual IR, a systematic large-scale…

Computation and Language · Computer Science 2025-09-19 Longfei Zuo , Pingjun Hong , Oliver Kraus , Barbara Plank , Robert Litschko

Large language models (LLMs) achieve remarkable advancements by leveraging tools to interact with environments, a critical step toward generalized AI. However, the standard supervised fine-tuning (SFT) approach, which relies on large-scale…

Computation and Language · Computer Science 2025-08-27 Junjie Ye , Yilong Wu , Sixian Li , Yuming Yang , Zhiheng Xi , Tao Gui , Qi Zhang , Xuanjing Huang , Peng Wang , Zhongchao Shi , Jianping Fan , Zhengyin Du

Large language models (LLMs) have demonstrated outstanding performance in natural language processing tasks. However, in the field of recommender systems, due to the inherent structural discrepancy between user behavior data and natural…

Information Retrieval · Computer Science 2026-01-01 Zekun Liu , Xiaowen Huang , Jitao Sang

Information retrieval (IR) plays a crucial role in locating relevant resources from vast amounts of data, and its applications have evolved from traditional knowledge bases to modern retrieval models (RMs). The emergence of large language…

Computation and Language · Computer Science 2023-12-13 Jiazhan Feng , Chongyang Tao , Xiubo Geng , Tao Shen , Can Xu , Guodong Long , Dongyan Zhao , Daxin Jiang

Large language models (LLMs) provide excellent text-generation capabilities, but standard prompting and generation methods generally do not lead to intentional or goal-directed agents and might necessitate considerable prompt tuning. This…

Computation and Language · Computer Science 2023-12-01 Marwa Abdulhai , Isadora White , Charlie Snell , Charles Sun , Joey Hong , Yuexiang Zhai , Kelvin Xu , Sergey Levine

Reward models (RMs) play a critical role in aligning large language models (LLMs) with human preferences. Yet in the domain of tool learning, the lack of RMs specifically designed for function-calling tasks has limited progress toward more…

Artificial Intelligence · Computer Science 2026-01-14 Renhao Li , Jianhong Tu , Yang Su , Yantao Liu , Fei Huang , Hamid Alinejad-Rokny , Derek F. Wong , Junyang Lin , Min Yang