中文
相关论文

相关论文: Exposing Limitations of Language Model Agents in S…

200 篇论文

This paper introduces the concept of Language-Guided World Models (LWMs) -- probabilistic models that can simulate environments by reading texts. Agents equipped with these models provide humans with more extensive and efficient control,…

计算与语言 · 计算机科学 2024-09-06 Alex Zhang , Khanh Nguyen , Jens Tuyls , Albert Lin , Karthik Narasimhan

The adoption of Large Language Models (LLMs) for code generation in data science offers substantial potential for enhancing tasks such as data manipulation, statistical analysis, and visualization. However, the effectiveness of these models…

Large language models (LLMs) have recently received considerable attention as alternative solutions for task planning. However, comparing the performance of language-oriented task planners becomes difficult, and there exists a dearth of…

人工智能 · 计算机科学 2024-02-14 Jae-Woo Choi , Youngwoo Yoon , Hyobin Ong , Jaehong Kim , Minsu Jang

Small language models (SLMs; 1-12B params, sometimes up to 20B) are sufficient and often superior for agentic workloads where the objective is schema- and API-constrained accuracy rather than open-ended generation. We synthesize recent…

人工智能 · 计算机科学 2025-10-07 Raghav Sharma , Manan Mehta

Modern Large Language Model (LLM) agents promise end to end assistance with real-world software tasks, yet existing benchmarks evaluate LLM agents almost exclusively in pre-baked environments where every dependency is pre-installed. To fill…

软件工程 · 计算机科学 2025-07-15 Avi Arora , Jinu Jang , Roshanak Zilouchian Moghaddam

Autonomous embodied agents live on an Internet of multimedia websites. Can they hop around multimodal websites to complete complex user tasks? Existing benchmarks fail to assess them in a realistic, evolving environment for their embodiment…

计算机视觉与模式识别 · 计算机科学 2025-07-01 Shulin Tian , Ziniu Zhang , Liangyu Chen , Ziwei Liu

While Large Language Models (LLMs) excel in language-based agentic tasks, their applicability to unseen, nonlinguistic environments (e.g., symbolic or spatial tasks) remains limited. Previous work attributes this performance gap to the…

人工智能 · 计算机科学 2026-02-03 Haoyu Wang , Guozheng Ma , Shugang Cui , Yilun Kong , Haotian Luo , Li Shen , Mengya Gao , Yichao Wu , Xiaogang Wang , Dacheng Tao

Large Language Models (LLMs) have demonstrated impressive capabilities across a range of scientific tasks including mathematics, physics, and chemistry. Despite their successes, the effectiveness of LLMs in handling complex statistical…

计算与语言 · 计算机科学 2024-10-11 Yizhang Zhu , Shiyin Du , Boyan Li , Yuyu Luo , Nan Tang

We introduce llama-embed-nemotron-8b, an open-weights text embedding model that achieves state-of-the-art performance on the Multilingual Massive Text Embedding Benchmark (MMTEB) leaderboard as of October 21, 2025. While recent models show…

计算与语言 · 计算机科学 2025-11-11 Yauhen Babakhin , Radek Osmulski , Ronay Ak , Gabriel Moreira , Mengyao Xu , Benedikt Schifferer , Bo Liu , Even Oldridge

Adapting Large Language Models (LLMs) to downstream tasks using Reinforcement Learning (RL) has proven to be an effective approach. However, LLMs do not inherently define the structure of an agent for RL training, particularly in terms of…

计算与语言 · 计算机科学 2025-03-28 Chengxing Jia , Ziniu Li , Pengyuan Wang , Yi-Chen Li , Zhenyu Hou , Yuxiao Dong , Yang Yu

Large language models (LLMs) are commonly adapted for diverse downstream tasks via parameter-efficient fine-tuning techniques such as Low-Rank Adapters (LoRA). While adapters can be combined to handle multiple tasks separately, standard…

Large language model (LLM) agents are increasingly used for complex tasks, yet deployed agents often remain static, failing to adapt as user needs evolve. This creates a tension between the need for continuous service and the necessity of…

Language models (LMs) for text data have been studied extensively for their usefulness in language generation and other downstream tasks. However, language modelling purely in the speech domain is still a relatively unexplored topic, with…

计算与语言 · 计算机科学 2021-11-02 Anurag Katakkar , Alan W Black

Although Multi-Agent Reinforcement Learning (MARL) is effective for complex multi-robot tasks, it suffers from low sample efficiency and requires iterative manual reward tuning. Large Language Models (LLMs) have shown promise in…

机器人学 · 计算机科学 2025-06-04 Guobin Zhu , Rui Zhou , Wenkang Ji , Shiyu Zhao

Automating the adaptation of software engineering (SE) research artifacts across datasets is essential for scalability and reproducibility, yet it remains largely unstudied. Recent advances in large language model (LLM)-based multi-agent…

软件工程 · 计算机科学 2025-11-27 Jingyi Chen , Xiaoyan Guo , Songqiang Chen , Shing-Chi Cheung , Jiasi Shen

Large Language Models (LLMs) have demonstrated remarkable capabilities in software engineering, yet comprehensive benchmarks covering diverse SE activities remain limited. We present a multi-task evaluation of 11 state-of-the-art LLMs…

软件工程 · 计算机科学 2026-02-10 Go Frendi Gunawan , Mukhlis Amien

Autonomous agents in safety-critical applications must continuously adapt to dynamic conditions without compromising performance and reliability. This work introduces TAPA (Training-free Adaptation of Programmatic Agents), a novel framework…

多智能体系统 · 计算机科学 2025-11-25 Jinwei Hu , Yi Dong , Youcheng Sun , Xiaowei Huang

Following the widespread adoption of ChatGPT in early 2023, numerous studies reported that large language models (LLMs) can match or even surpass human performance in creative tasks. However, it remains unclear whether LLMs have become more…

计算与语言 · 计算机科学 2025-04-18 Jennifer Haase , Paul H. P. Hanel , Sebastian Pokutta

Language models (LMs) are often expected to generate strings in some formal language; for example, structured data, API calls, or code snippets. Although LMs can be tuned to improve their adherence to formal syntax, this does not guarantee…

计算与语言 · 计算机科学 2024-08-06 Terry Koo , Frederick Liu , Luheng He

Large language models (LLMs) are rapidly approaching the level of proficiency in university-level symbolic mathematics required for applications in advanced science and technology. However, existing benchmarks fall short in assessing the…

计算与语言 · 计算机科学 2025-06-02 Michael Shalyt , Rotem Elimelech , Ido Kaminer