English
Related papers

Related papers: Automating Steering for Safe Multimodal Large Lang…

200 papers

Multimodal Large Reasoning Models (MLRMs) demonstrate impressive cross-modal reasoning but often amplify safety risks under adversarial or unsafe prompts, a phenomenon we call the \textit{Reasoning Tax}. Existing defenses mainly act at the…

Machine Learning · Computer Science 2025-10-10 Huahui Yi , Kun Wang , Qiankun Li , Miao Yu , Liang Lin , Gongli Xi , Hao Wu , Xuming Hu , Kang Li , Yang Liu

Recent advancements in Large Language Models (LLMs) offer new opportunities to create natural language interfaces for Autonomous Driving Systems (ADSs), moving beyond rigid inputs. This paper addresses the challenge of mapping the…

Robotics · Computer Science 2026-01-26 Marvin Seegert , Korbinian Moller , Johannes Betz

Traditional approaches to safety event analysis in autonomous systems have relied on complex machine learning models and extensive datasets for high accuracy and reliability. However, the advent of Multimodal Large Language Models (MLLMs)…

Computer Vision and Pattern Recognition · Computer Science 2024-06-21 Mohammad Abu Tami , Huthaifa I. Ashqar , Mohammed Elhenawy

Traffic safety remains a critical global challenge, with traditional Advanced Driver-Assistance Systems (ADAS) often struggling in dynamic real-world scenarios due to fragmented sensor processing and susceptibility to adversarial…

Computer Vision and Pattern Recognition · Computer Science 2025-04-24 Mohammad Abu Tami , Mohammed Elhenawy , Huthaifa I. Ashqar

Multimodal large language models (MLLMs) have demonstrated remarkable capabilities across vision-language tasks, yet their large-scale deployment raises pressing concerns about memorized private data, outdated knowledge, and harmful…

Machine Learning · Computer Science 2026-02-03 Chenlu Ding , Jiancan Wu , Leheng Sheng , Fan Zhang , Yancheng Yuan , Xiang Wang , Xiangnan He

Sparse Autoencoders (SAEs) can extract interpretable features from large language models (LLMs) without supervision. However, their effectiveness in downstream steering tasks is limited by the requirement for contrastive datasets or large…

Computation and Language · Computer Science 2026-05-05 Seonglae Cho , Zekun Wu , Adriano Koshiyama

Automated Machine Learning (AutoML) offers a promising approach to streamline the training of machine learning models. However, existing AutoML frameworks are often limited to unimodal scenarios and require extensive manual configuration.…

Machine Learning · Computer Science 2024-08-02 Daqin Luo , Chengjian Feng , Yuxuan Nong , Yiqing Shen

As large language models (LLMs) become more integrated into societal systems, the risk of them perpetuating and amplifying harmful biases becomes a critical safety concern. Traditional methods for mitigating bias often rely on data…

Artificial Intelligence · Computer Science 2025-08-13 Shivam Dubey

Recent work on activation and latent steering has demonstrated that modifying internal representations can effectively guide large language models (LLMs) toward improved reasoning and efficiency without additional training. However, most…

Machine Learning · Computer Science 2026-01-07 Tuc Nguyen , Thai Le

In this era of technological advancements, several cutting-edge techniques are being implemented to enhance Autonomous Driving (AD) systems, focusing on improving safety, efficiency, and adaptability in complex driving environments.…

Computation and Language · Computer Science 2025-02-27 Md Robiul Islam

Activation steering methods enable inference-time control of large language model (LLM) behavior without retraining, but current approaches face a fundamental trade-off: sample-efficient methods suboptimally capture steering signals from…

Machine Learning · Computer Science 2026-03-09 Kartik Sharma , Rakshit S. Trivedi

Precise control over language model generation is vital for ensuring both safety and reliability. Although prompt engineering and steering are commonly used to intervene in model behaviors, the vast number of parameters in models often…

Computation and Language · Computer Science 2025-06-04 Mengru Wang , Ziwen Xu , Shengyu Mao , Shumin Deng , Zhaopeng Tu , Huajun Chen , Ningyu Zhang

As LLMs are increasingly deployed in real-world applications, ensuring their ability to refuse malicious prompts, especially jailbreak attacks, is essential for safe and reliable use. Recently, activation steering has emerged as an…

Machine Learning · Computer Science 2026-02-10 Leheng Sheng , Changshuo Shen , Weixiang Zhao , Junfeng Fang , Xiaohao Liu , Zhenkai Liang , Xiang Wang , An Zhang , Tat-Seng Chua

Aligning Large Language Models (LLMs) with specific personas typically relies on expensive and monolithic Supervised Fine-Tuning (SFT) or RLHF. While effective, these methods require training distinct models for every target personality…

Computation and Language · Computer Science 2026-03-05 Florian Hoppe , David Khachaturov , Robert Mullins , Mark Huasong Meng

Human drivers possess spatial and causal intelligence, enabling them to perceive driving scenarios, anticipate hazards, and react to dynamic environments. In contrast, autonomous vehicles lack these abilities, making it challenging to…

Robotics · Computer Science 2025-09-12 Shucheng Huang , Freda Shi , Chen Sun , Jiaming Zhong , Minghao Ning , Yufeng Yang , Yukun Lu , Hong Wang , Amir Khajepour

We introduce Mechanistic Error Reduction with Abstention (MERA), a principled framework for steering language models (LMs) to mitigate errors through selective, adaptive interventions. Unlike existing methods that rely on fixed, manually…

Machine Learning · Computer Science 2025-10-16 Anna Hedström , Salim I. Amoukou , Tom Bewley , Saumitra Mishra , Manuela Veloso

Ensuring the safety and harmlessness of Large Language Models (LLMs) has become equally critical as their performance in applications. However, existing safety alignment methods typically suffer from safety-performance trade-offs and the…

Computation and Language · Computer Science 2025-06-30 Yichi Zhang , Siyuan Zhang , Yao Huang , Zeyu Xia , Zhengwei Fang , Xiao Yang , Ranjie Duan , Dong Yan , Yinpeng Dong , Jun Zhu

Agentic methods have emerged as a powerful and autonomous paradigm that enhances reasoning, collaboration, and adaptive control, enabling systems to coordinate and independently solve complex tasks. We extend this paradigm to safety…

Artificial Intelligence · Computer Science 2025-10-30 Juan Ren , Mark Dras , Usman Naseem

Large Language Models (LLMs) have emerged as powerful tools, but their inherent safety risks - ranging from harmful content generation to broader societal harms - pose significant challenges. These risks can be amplified by the recent…

Recent advancements in large language models (LLMs) have accelerated progress toward artificial general intelligence, yet their potential to generate harmful content poses critical safety challenges. Existing alignment methods often…

Computation and Language · Computer Science 2025-10-08 Kehua Feng , Keyan Ding , Yuhao Wang , Menghan Li , Fanjunduo Wei , Xinda Wang , Qiang Zhang , Huajun Chen