中文
相关论文

相关论文: Evaluating LLM Reasoning Beyond Correctness and Co…

200 篇论文

Large Language Models (LLMs) have significantly advanced software engineering (SE) tasks, with prompt engineering techniques enhancing their performance in code-related areas. However, the rapid development of foundational LLMs such as the…

软件工程 · 计算机科学 2024-11-05 Guoqing Wang , Zeyu Sun , Zhihao Gong , Sixiang Ye , Yizhou Chen , Yifan Zhao , Qingyuan Liang , Dan Hao

Reasoning-augmented vision language models (VLMs) generate explicit chains of thought that promise greater capability and transparency but also introduce new failure modes: models may reach correct answers via visually unfaithful…

计算机视觉与模式识别 · 计算机科学 2025-12-22 Rheeya Uppaal , Phu Mon Htut , Min Bai , Nikolaos Pappas , Zheng Qi , Sandesh Swamy

We pursue a vision for self-improving language models in which the model does not merely generate problems or traces to imitate, but constructs the environments that train it. In zero-data reasoning RL, this reframes self-improvement from a…

人工智能 · 计算机科学 2026-05-15 Yucheng Shi , Zhenwen Liang , Kishan Panaganti , Dian Yu , Wenhao Yu , Haitao Mi

Exceptional mathematical reasoning ability is one of the key features that demonstrate the power of large language models (LLMs). How to comprehensively define and evaluate the mathematical abilities of LLMs, and even reflect the user…

计算与语言 · 计算机科学 2024-10-10 Zihao Zhou , Shudong Liu , Maizhen Ning , Wei Liu , Jindong Wang , Derek F. Wong , Xiaowei Huang , Qiufeng Wang , Kaizhu Huang

Large language models (LLMs) have made remarkable strides in complex reasoning tasks, but their safety and robustness in reasoning processes remain underexplored. Existing attacks on LLM reasoning are constrained by specific settings or…

人工智能 · 计算机科学 2025-06-17 Jingyu Peng , Maolin Wang , Xiangyu Zhao , Kai Zhang , Wanyu Wang , Pengyue Jia , Qidong Liu , Ruocheng Guo , Qi Liu

Large language models (LLMs) have achieved impressive performance across various mathematical reasoning benchmarks. However, there are increasing debates regarding whether these models truly understand and apply mathematical knowledge or…

计算与语言 · 计算机科学 2024-07-03 Qintong Li , Leyang Cui , Xueliang Zhao , Lingpeng Kong , Wei Bi

Large Language Models (LLMs) have shown impressive performance on complex tasks through Chain-of-Thought (CoT) reasoning. However, conventional CoT relies on explicitly verbalized intermediate steps, which constrains its broader…

计算与语言 · 计算机科学 2025-11-04 Xinghao Chen , Anhao Zhao , Heming Xia , Xuan Lu , Hanlin Wang , Yanjun Chen , Wei Zhang , Jian Wang , Wenjie Li , Xiaoyu Shen

Vision-language models (VLMs) have recently demonstrated strong efficacy as visual assistants that can parse natural queries about the visual content and generate human-like outputs. In this work, we explore the ability of these models to…

计算与语言 · 计算机科学 2024-03-21 Yangyi Chen , Karan Sikka , Michael Cogswell , Heng Ji , Ajay Divakaran

Recent advancements in Large Language Models (LLMs) have sparked interest in their formal reasoning capabilities, particularly in mathematics. The GSM8K benchmark is widely used to assess the mathematical reasoning of models on…

机器学习 · 计算机科学 2025-08-28 Iman Mirzadeh , Keivan Alizadeh , Hooman Shahrokhi , Oncel Tuzel , Samy Bengio , Mehrdad Farajtabar

Large language models (LLMs) face significant challenges in effectively leveraging sequential environmental feedback (EF) signals, such as natural language evaluations, for feedback-independent chain-of-thought (CoT) reasoning. Existing…

计算与语言 · 计算机科学 2025-07-29 Kang Yang , Jingxue Chen , Qingkun Tang , Tianxiang Zhang , Qianchun Lu

When writing and talking, people sometimes pause to think. Although reasoning-focused works have often framed reasoning as a method of answering questions or completing agentic tasks, reasoning is implicit in almost all written text. For…

计算与语言 · 计算机科学 2024-03-19 Eric Zelikman , Georges Harik , Yijia Shao , Varuna Jayasiri , Nick Haber , Noah D. Goodman

Large language models (LLMs) demonstrate impressive capabilities in mathematical reasoning. However, despite these achievements, current evaluations are mostly limited to specific mathematical topics, and it remains unclear whether LLMs are…

计算与语言 · 计算机科学 2025-04-01 Arash Gholami Davoodi , Seyed Pouyan Mousavi Davoudi , Pouya Pezeshkpour

Recent advances in large language models (LLMs) have demonstrated impressive reasoning capacities that mirror human-like thinking. However, whether LLMs possess genuine fluid intelligence (i.e., the ability to reason abstractly and…

Generating accurate step-by-step reasoning is essential for Large Language Models (LLMs) to address complex problems and enhance robustness and interpretability. Despite the flux of research on developing advanced reasoning approaches,…

Prior work has combined chain-of-thought prompting in large language models (LLMs) with programmatic representations to perform effective and transparent reasoning. While such an approach works well for tasks that only require forward…

计算与语言 · 计算机科学 2023-10-13 Xi Ye , Qiaochu Chen , Isil Dillig , Greg Durrett

Chain-of-thought (CoT) has proven to improve the reasoning capability of large language models (LLMs). However, due to the complexity of multimodal scenarios and the difficulty in collecting high-quality CoT data, CoT reasoning in…

机器学习 · 计算机科学 2024-11-05 Kanzhi Cheng , Yantao Li , Fangzhi Xu , Jianbing Zhang , Hao Zhou , Yang Liu

Large language models (LLMs) solve complex problems by generating multi-step reasoning traces. Yet these traces are typically analyzed from only one of two perspectives: the sequence of tokens across different reasoning steps in the…

计算与语言 · 计算机科学 2026-03-25 Ruidi Chang , Jiawei Zhou , Hanjie Chen

We present Team asdfo123's submission to the LLMSR@XLLM25 shared task, which evaluates large language models on producing fine-grained, controllable, and interpretable reasoning processes. Systems must extract all problem conditions,…

计算与语言 · 计算机科学 2025-05-20 Xinye Li , Mingqi Wan , Dianbo Sui

Formal logic enables computers to reason in natural language by representing sentences in symbolic forms and applying rules to derive conclusions. However, in what our study characterizes as "rulebreaker" scenarios, this method can lead to…

计算与语言 · 计算机科学 2025-08-18 Jason Chan , Robert Gaizauskas , Zhixue Zhao

Large Language Models (LLMs) have demonstrated impressive capabilities in structured reasoning and symbolic tasks, with coding emerging as a particularly successful application. This progress has naturally motivated efforts to extend these…

人工智能 · 计算机科学 2026-02-02 Andrea Asperti , Alberto Naibo , Claudio Sacerdoti Coen