English
Related papers

Related papers: Detecting Proxy Gaming in RL and LLM Alignment via…

200 papers

We propose a novel LLM-based framework for reasoning in discrete, game-theoretic tasks, illustrated with \emph{Tic-Tac-Toe}. The method integrates in-context learning with entropy-guided chain-of-thought (CoT) reasoning and adaptive context…

Computation and Language · Computer Science 2026-04-14 Tommaso Felice Banfi , Sashenka Gamage

While advancements in NLP have significantly improved the performance of Large Language Models (LLMs) on tasks requiring vertical thinking, their lateral thinking capabilities remain under-explored and challenging to measure due to the…

Computation and Language · Computer Science 2024-10-10 Qi Chen , Bowen Zhang , Gang Wang , Qi Wu

In this paper, we present a challenging code reasoning task: vulnerability detection. Large Language Models (LLMs) have shown promising results in natural-language and math reasoning, but state-of-the-art (SOTA) models reported only 54.5%…

Software Engineering · Computer Science 2025-01-09 Benjamin Steenhoek , Md Mahbubur Rahman , Monoshi Kumar Roy , Mirza Sanjida Alam , Hengbo Tong , Swarna Das , Earl T. Barr , Wei Le

Deep Reinforcement Learning (DRL) algorithms have been successfully applied to a range of challenging control tasks. However, these methods typically suffer from three core difficulties: temporal credit assignment with sparse rewards, lack…

Machine Learning · Computer Science 2018-10-30 Shauharda Khadka , Kagan Tumer

This study reveals how frontier Large Language Models LLMs can "game the system" when faced with impossible situations, a critical security and alignment concern. Using a novel textual simulation approach, we presented three leading LLMs…

Artificial Intelligence · Computer Science 2025-05-14 Lars Malmqvist

While scaling laws optimize training configurations for large language models (LLMs) through experiments on smaller or early-stage models, they fail to predict emergent abilities due to the absence of such capabilities in these models. To…

Computation and Language · Computer Science 2024-12-11 Bo-Wen Zhang , Yan Yan , Boxiang Yang , Yifei Xue , Guang Liu

Ensuring the safety and harmlessness of Large Language Models (LLMs) has become equally critical as their performance in applications. However, existing safety alignment methods typically suffer from safety-performance trade-offs and the…

Computation and Language · Computer Science 2025-06-30 Yichi Zhang , Siyuan Zhang , Yao Huang , Zeyu Xia , Zhengwei Fang , Xiao Yang , Ranjie Duan , Dong Yan , Yinpeng Dong , Jun Zhu

As large language models (LLMs) improve, so do their offensive applications: frontier agents now generate working exploits for under $50 in compute (Heelan, 2026). Defensive incident response (IR) agents must keep pace, but existing…

Artificial Intelligence · Computer Science 2026-02-10 Jarrod Barnes

The standard Reinforcement Learning from Human Feedback (RLHF) framework primarily focuses on optimizing the performance of large language models using pre-collected prompts. However, collecting prompts that provide comprehensive coverage…

Computation and Language · Computer Science 2024-06-18 Rui Zheng , Hongyi Guo , Zhihan Liu , Xiaoying Zhang , Yuanshun Yao , Xiaojun Xu , Zhaoran Wang , Zhiheng Xi , Tao Gui , Qi Zhang , Xuanjing Huang , Hang Li , Yang Liu

Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language understanding and generation, enabling their widespread adoption across various domains. However, their susceptibility to prompt injection attacks…

Cryptography and Security · Computer Science 2025-05-05 Jinsheng Pan , Xiaogeng Liu , Chaowei Xiao

Adversarial board games, as a paradigmatic domain of strategic reasoning and intelligence, have long served as both a popular competitive activity and a benchmark for evaluating artificial intelligence (AI) systems. Building on this…

Artificial Intelligence · Computer Science 2025-08-08 Yingjie Zhou , Jiezhang Cao , Farong Wen , Li Xu , Yanwei Jiang , Jun Jia , Ronghui Li , Xiaohong Liu , Yu Zhou , Xiongkuo Min , Jie Guo , Zicheng Zhang , Guangtao Zhai

Large language models (LLMs) are increasingly used as judges to evaluate agent performance, particularly in non-verifiable settings where judgments rely on agent trajectories including chain-of-thought (CoT) reasoning. This paradigm…

Artificial Intelligence · Computer Science 2026-01-23 Muhammad Khalifa , Lajanugen Logeswaran , Jaekyeom Kim , Sungryull Sohn , Yunxiang Zhang , Moontae Lee , Hao Peng , Lu Wang , Honglak Lee

Large Language Models (LLMs) have achieved remarkable success across domains such as healthcare, education, and cybersecurity. However, this openness also introduces significant security risks, particularly through embedding space…

Computation and Language · Computer Science 2025-07-14 Zhibo Zhang , Yuxi Li , Kailong Wang , Shuai Yuan , Ling Shi , Haoyu Wang

As reinforcement Learning with Verifiable Rewards (RLVR) has become the dominant paradigm for scaling reasoning capabilities in LLMs, a new failure mode emerges: LLMs gaming verifiers. We study this phenomenon on inductive reasoning tasks,…

Large Language Models (LLMs) are increasingly relied upon to evaluate text outputs of other LLMs, thereby influencing leaderboards and development decisions. However, concerns persist over the accuracy of these assessments and the potential…

Computation and Language · Computer Science 2024-11-27 Sumanth Doddapaneni , Mohammed Safi Ur Rahman Khan , Sshubam Verma , Mitesh M. Khapra

Large Language Models (LLMs) excel in reasoning and generation across domains, but still struggle with identifying and diagnosing complex errors. This stems mainly from training objectives that prioritize correct answers, limiting exposure…

Artificial Intelligence · Computer Science 2025-08-06 Rui Zou , Mengqi Wei , Yutao Zhu , Jirong Wen , Xin Zhao , Jing Chen

LLM-based multi-agent systems (MAS) show promise on complex tasks but remain prone to coordination failures such as goal drift, error cascades, and misaligned behaviors. We propose Explicit Trait Inference (ETI), a psychologically grounded…

Artificial Intelligence · Computer Science 2026-04-23 Suhaib Abdurahman , Etsuko Ishii , Katerina Margatina , Divya Bhargavi , Monica Sunkara , Yi Zhang

LLM agents increasingly operate in open-ended environments spanning hundreds of sequential episodes, yet they remain largely stateless: each task is solved from scratch without converting past experience into better future behavior. The…

Computation and Language · Computer Science 2026-04-24 Wujiang Xu , Jiaojiao Han , Minghao Guo , Kai Mei , Xi Zhu , Han Zhang , Dimitris N. Metaxas

Given that Large Language Models (LLMs) are increasingly applied to automate software development, comprehensive software assurance spans three distinct goals: regression prevention, reactive reproduction, and proactive discovery. Current…

Software Engineering · Computer Science 2026-02-24 Steven Liu , Jane Luo , Xin Zhang , Aofan Liu , Hao Liu , Jie Wu , Ziyang Huang , Yangyu Huang , Yu Kang , Scarlett Li

Generative reward models (GRMs) for vision-language models (VLMs) often evaluate outputs via a three-stage pipeline: rubric generation, criterion-based scoring, and a final verdict. However, the intermediate rubric is rarely optimized…

Computer Vision and Pattern Recognition · Computer Science 2026-03-19 Weijie Qiu , Dai Guan , Junxin Wang , Zhihang Li , Yongbo Gai , Mengyu Zhou , Erchao Zhao , Xiaoxi Jiang , Guanjun Jiang