English
Related papers

Related papers: Sycophancy Mitigation Through Reinforcement Learni…

200 papers

Reinforcement learning (RL) for mathematical reasoning can suffer from reward sparsity: for challenging problems, LLM fails to sample any correct trajectories, preventing RL from receiving meaningful positive feedback. At the same time,…

Machine Learning · Computer Science 2026-03-06 Yangzhen Wu , Shanda Li , Zixin Wen , Xin Zhou , Ameet Talwalkar , Yiming Yang , Wenhao Huang , Tianle Cai

Large language models (LLMs) have demonstrated strong reasoning abilities in mathematical tasks, often enhanced through reinforcement learning (RL). However, RL-trained models frequently produce unnecessarily long reasoning traces -- even…

Computation and Language · Computer Science 2025-05-27 Jinyan Su , Claire Cardie

Large Language Models (LLMs) have achieved remarkable performance across a wide range of mathematical benchmarks. However, concerns remain as to whether these successes reflect genuine reasoning or superficial pattern recognition. Existing…

Artificial Intelligence · Computer Science 2026-04-21 Yujie Hou , Mei Wang , Yaoyao Zhong , Ting Zhang , Xuetao Ma , Hua Huang

Chain-of-Thought (CoT) has significantly enhanced the reasoning capabilities of Large Language Models (LLMs), especially when combined with reinforcement learning (RL) based post-training methods. While longer reasoning traces can improve…

Machine Learning · Computer Science 2026-02-16 Qinhang Wu , Sen Lin , Ming Zhang , Yingbin Liang , Ness B. Shroff

Reinforcement learning (RL) has proven effective in incentivizing the reasoning abilities of large language models (LLMs), but suffers from severe efficiency challenges due to its trial-and-error nature. While the common practice employs…

Computation and Language · Computer Science 2025-10-17 Liang Chen , Xueting Han , Li Shen , Jing Bai , Kam-Fai Wong

Reinforcement Learning with Verifiable Rewards (RLVR) has established itself as the dominant paradigm for instilling rigorous reasoning capabilities in Large Language Models. While effective at amplifying dominant behaviors, we identify a…

Machine Learning · Computer Science 2026-02-16 Zesheng Hong , Jiadong Yu , Hui Pan

Multi-vehicle trajectory planning is a non-convex problem that becomes increasingly difficult in dense environments due to the rapid growth of collision constraints. Efficient exploration of feasible behaviors and resolution of tight…

Robotics · Computer Science 2025-09-22 Heye Huang , Yibin Yang , Wang Chen , Tiantian Chen , Xiaopeng Li , Sikai Chen

Test-time adaptation offers a promising avenue for improving reasoning performance in large language models without additional supervision, but existing approaches often apply a uniform optimization objective across all inputs, leading to…

Computation and Language · Computer Science 2026-03-06 Mohammad Mahdi Moradi , Sudhir Mudur

The increase in computing power and the necessity of AI-assisted decision-making boost the growing application of large language models (LLMs). Along with this, the potential retention of sensitive data of LLMs has spurred increasing…

Computation and Language · Computer Science 2026-04-20 Chenchen Tan , Youyang Qu , Xinghao Li , Hui Zhang , Shujie Cui , Cunjian Chen , Longxiang Gao

While Large Reasoning Models (LRMs) have achieved remarkable performance by scaling test-time compute, they frequently suffer from Cognitive Inertia, a failure pattern manifesting as either overthinking (inertia of motion) or reasoning…

Machine Learning · Computer Science 2026-02-02 Seojin Lee , ByeongJeong Kim , Hwanhee Lee

Large language models (LLMs) have achieved remarkable progress in reasoning tasks, yet the optimal integration of Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) remains a fundamental challenge. Through comprehensive analysis…

Computation and Language · Computer Science 2025-06-25 Yuqian Fu , Tinghong Chen , Jiajun Chai , Xihuai Wang , Songjun Tu , Guojun Yin , Wei Lin , Qichao Zhang , Yuanheng Zhu , Dongbin Zhao

AI tools in pathology have improved screening throughput, standardized quantification, and revealed prognostic patterns that inform treatment. However, adoption remains limited because most systems still lack the human-readable reasoning…

Artificial Intelligence · Computer Science 2025-11-18 Yunqi Hong , Johnson Kao , Liam Edwards , Nein-Tzu Liu , Chung-Yen Huang , Alex Oliveira-Kowaleski , Cho-Jui Hsieh , Neil Y. C. Lin

Rapid improvements in large language models have unveiled a critical challenge in human-AI interaction: sycophancy. In this context, sycophancy refers to the tendency of models to excessively agree with or flatter users, often at the…

Computation and Language · Computer Science 2025-03-18 Joshua Liu , Aarav Jain , Soham Takuri , Srihan Vege , Aslihan Akalin , Kevin Zhu , Sean O'Brien , Vasu Sharma

Sycophancy (overly agreeable or flattering behavior) poses a fundamental challenge for human-AI collaboration, particularly in high-stakes decision-making domains such as health, law, and education. A central difficulty in studying…

Artificial Intelligence · Computer Science 2026-05-05 Katherine Atwell , Pedram Heydari , Anthony Sicilia , Malihe Alikhani

Table reasoning with large language models (LLMs) plays a critical role in building intelligent systems capable of understanding and analyzing tabular data. Despite recent progress, existing methods still face key limitations: their…

Artificial Intelligence · Computer Science 2026-01-27 Huajian Zhang , Mingyue Cheng , Yucong Luo , Xiaoyu Tao

Reasoning methods that adaptively allocate test-time compute have advanced LLM performance on easy to verify domains such as math and code. In this work, we study how to utilize this approach to train models that exhibit a degree of…

Machine Learning · Computer Science 2025-10-28 Taeyoun Kim , Fahim Tajwar , Aditi Raghunathan , Aviral Kumar

Large reasoning models (LRMs) achieve strong performance through extended reasoning traces, but they often exhibit overthinking behavior for low-complexity queries. Existing efforts to mitigate this issue are fundamentally limited by…

Machine Learning · Computer Science 2026-02-27 Zihang Xu , Haozhi Xie , Ziqi Miao , Wuxuan Gong , Chen Qian , Lijun Li

Large language models (LLMs) typically operate in a question-answering paradigm, where the quality of the input prompt critically affects the response. Automated Prompt Optimization (APO) aims to overcome the cognitive biases of manually…

Computation and Language · Computer Science 2025-11-13 Jian Zhang , Zhangqi Wang , Haiping Zhu , Kangda Cheng , Kai He , Bo Li , Qika Lin , Jun Liu , Erik Cambria

The inherent capabilities of a language model (LM) and the reasoning strategies it employs jointly determine its performance in reasoning tasks. While test-time scaling is regarded as an effective approach to tackling complex reasoning…

Computation and Language · Computer Science 2025-05-27 Zhihong Pan , Kai Zhang , Yuze Zhao , Yupeng Han

OpenAI o1 has shown that applying reinforcement learning to integrate reasoning steps directly during inference can significantly improve a model's reasoning capabilities. This result is exciting as the field transitions from the…

Artificial Intelligence · Computer Science 2025-02-18 Jun Wang