English
Related papers

Related papers: GuardTrace-VL: Detecting Unsafe Multimodel Reasoni…

200 papers

The emergence of Large Reasoning Models (LRMs) introduces a new paradigm of explicit reasoning, enabling remarkable advances yet posing unique risks such as reasoning manipulation and information leakage. To mitigate these risks, current…

Artificial Intelligence · Computer Science 2026-02-03 Jingnan Zheng , Jingjun Xu , Yanzhen Luo , Chenhang Cui , Gelei Deng , Zhenkai Liang , Xiang Wang , An Zhang , Tat-Seng Chua

Accurate rejection of sensitive or harmful visual content, i.e., harmful image guardrail, is critical in many application scenarios. This task must continuously adapt to the evolving safety policies and content across various domains and…

Computer Vision and Pattern Recognition · Computer Science 2026-04-01 Caiyong Piao , Zhiyuan Yan , Haoming Xu , Yunzhen Zhao , Kaiqing Lin , Feiyang Xu , Shuigeng Zhou

Large Language Models (LLMs) are vulnerable to jailbreak attacks that exploit weaknesses in traditional safety alignment, which often relies on rigid refusal heuristics or representation engineering to block harmful outputs. While they are…

Computation and Language · Computer Science 2025-10-01 Yuyou Zhang , Miao Li , William Han , Yihang Yao , Zhepeng Cen , Ding Zhao

The sequential structure of videos poses a challenge to the ability of multimodal large language models (MLLMs) to locate multi-frame evidence and conduct multimodal reasoning. However, existing video benchmarks mainly focus on…

Computer Vision and Pattern Recognition · Computer Science 2025-06-05 Kejian Zhu , Zhuoran Jin , Hongbang Yuan , Jiachun Li , Shangqing Tu , Pengfei Cao , Yubo Chen , Kang Liu , Jun Zhao

With the integration of an additional modality, large vision-language models (LVLMs) exhibit greater vulnerability to safety risks (e.g., jailbreaking) compared to their language-only predecessors. Although recent studies have devoted…

Machine Learning · Computer Science 2025-01-07 Ziwei Zheng , Junyao Zhao , Le Yang , Lijun He , Fan Li

Large Language Models (LLMs) face threats from jailbreak prompts. Existing methods for detecting jailbreak prompts are primarily online moderation APIs or finetuned LLMs. These strategies, however, often require extensive and…

Computation and Language · Computer Science 2024-05-31 Yueqi Xie , Minghong Fang , Renjie Pi , Neil Gong

Large Reasoning Models (LRMs) have demonstrated remarkable performance on complex tasks by engaging in extended reasoning before producing final answers. Beyond improving abilities, these detailed reasoning traces also create a new…

Computation and Language · Computer Science 2026-01-08 Shu Yang , Junchao Wu , Xilin Gong , Xuansheng Wu , Derek Wong , Ninghao Liu , Di Wang

Eye-tracking data reveals valuable insights into users' cognitive states but is difficult to analyze due to its structured, non-linguistic nature. While large language models (LLMs) excel at reasoning over text, they struggle with temporal…

Human-Computer Interaction · Computer Science 2025-07-25 Dongyang Guo , Yasmeen Abdrabou , Enkeleda Thaqi , Enkelejda Kasneci

Large language models (LLMs) are increasingly embedded in Computer Science (CS) classrooms to automate code generation, feedback, and assessment. However, their susceptibility to adversarial or ill-intentioned prompts threatens student…

Computers and Society · Computer Science 2026-02-04 Nishat Raihan , Noah Erdachew , Jayoti Devi , Joanna C. S. Santos , Marcos Zampieri

Recent progress in Multi-modal Large Language Models (MLLMs) has enabled step-by-step multi-modal mathematical reasoning by performing visual operations based on the textual instructions. A promising approach uses code as an intermediate…

Computation and Language · Computer Science 2025-11-06 Xiaoyuan Li , Moxin Li , Wenjie Wang , Rui Men , Yichang Zhang , Fuli Feng , Dayiheng Liu

Knowledge-intensive visual question answering (VQA) requires external knowledge beyond image content, demanding precise visual grounding and coherent integration of visual and textual information. Although multimodal retrieval-augmented…

Computer Vision and Pattern Recognition · Computer Science 2026-04-23 Changin Choi , Wonseok Lee , Jungmin Ko , Wonjong Rhee

While explicit Chain-of-Thought (CoT) empowers large reasoning models (LRMs), it enables the generation of riskier final answers. Current alignment paradigms primarily rely on externally enforced compliance, optimizing models to detect…

Artificial Intelligence · Computer Science 2026-05-12 Yi Zhang , Yuxin Chen , Leheng Sheng , Dongcheng Zhang , Chaochao Lu , Xiang Wang , An Zhang

Multimodal large language models (MLLMs) have become integral to a wide range of real-world applications by jointly reasoning over text and visual inputs. However, despite recent advances in safety alignment, MLLMs remain vulnerable to…

Cryptography and Security · Computer Science 2026-03-10 Xinkai Wang , Beibei Li , Zerui Shao , Ao Liu , Guangquan Xu , Shouling Ji

Multimodal large language models (MLLMs) show promise in tasks like visual question answering (VQA) but still face challenges in multimodal reasoning. Recent works adapt agentic frameworks or chain-of-thought (CoT) reasoning to improve…

Artificial Intelligence · Computer Science 2025-03-12 Zhuo Zhi , Chen Feng , Adam Daneshmend , Mine Orlu , Andreas Demosthenous , Lu Yin , Da Li , Ziquan Liu , Miguel R. D. Rodrigues

Multimodal large language models are increasingly expected to perform thinking with images, yet existing visual latent reasoning methods still rely on explicit textual chain-of-thought interleaved with visual latent tokens. This interleaved…

Computer Vision and Pattern Recognition · Computer Science 2026-05-13 Houcheng Jiang , Jiajun Fu , Junfeng Fang , Chen Gao , Xiang Wang , Xiangnan He , Yong Li

Enhancing the multimodal reasoning capabilities of Multimodal Large Language Models (MLLMs) is a challenging task that has attracted increasing attention in the community. Recently, several studies have applied Reinforcement Learning with…

Machine Learning · Computer Science 2026-03-04 Tong Xiao , Xin Xu , Zhenya Huang , Hongyu Gao , Quan Liu , Qi Liu , Enhong Chen

Large Vision-Language Models (LVLMs) have made remarkable strides in multimodal tasks such as visual question answering, visual grounding, and complex reasoning. However, they remain limited by static training data, susceptibility to…

Artificial Intelligence · Computer Science 2025-08-27 Chan-Wei Hu , Yueqi Wang , Shuo Xing , Chia-Ju Chen , Suofei Feng , Ryan Rossi , Zhengzhong Tu

The trustworthiness of Multimodal Large Language Models (MLLMs) remains an intense concern despite the significant progress in their capabilities. Existing evaluation and mitigation approaches often focus on narrow aspects and overlook…

Computation and Language · Computer Science 2025-08-22 Yichi Zhang , Yao Huang , Yifan Wang , Yitong Sun , Chang Liu , Zhe Zhao , Zhengwei Fang , Huanran Chen , Xiao Yang , Xingxing Wei , Hang Su , Yinpeng Dong , Jun Zhu

Visual reasoning is central to human cognition, enabling individuals to interpret and abstractly understand their environment. Although recent Multimodal Large Language Models (MLLMs) have demonstrated impressive performance across language…

Computer Vision and Pattern Recognition · Computer Science 2025-03-17 Jing Bi , Junjia Guo , Susan Liang , Guangyu Sun , Luchuan Song , Yunlong Tang , Jinxi He , Jiarui Wu , Ali Vosoughi , Chen Chen , Chenliang Xu

Multimodal large language models (MLLMs) are gaining increasing attention. Due to the heterogeneity of their input features, they face significant challenges in terms of jailbreak defenses. Current defense methods rely on costly fine-tuning…

Artificial Intelligence · Computer Science 2026-05-13 Xinyi Zeng , Xue Yang , Jingyuan Zhang , Huanqian Yan , Xiang Chen , Kaiwen Wei , Hankun Kang , Yu Tian