English
Related papers

Related papers: Adversarial attacks against Modern Vision-Language…

200 papers

Vision Language Model (VLM)-based agents are stateful, autonomous entities capable of perceiving and interacting with their environments through vision and language. Multi-agent systems comprise specialized agents who collaborate to solve a…

Multiagent Systems · Computer Science 2025-08-14 Yutong Wu , Jie Zhang , Yiming Li , Chao Zhang , Qing Guo , Nils Lukas , Tianwei Zhang

Large Language Model (LLM) agents remain vulnerable to safety threats from the external environment, where attackers inject adversarial content into external observations such as tool-returned data, webpages, or MCP context, causing harmful…

Artificial Intelligence · Computer Science 2026-05-28 Yongxiang Li , Moxin Li , Zhixin Ma , Fengbin Zhu , Dongrui Liu , Wenjie Wang , Fuli Feng

Modern vision-language-model (VLM) based graphical user interface (GUI) agents are expected not only to execute actions accurately but also to respond to user instructions with low latency. While existing research on GUI-agent security…

Cryptography and Security · Computer Science 2026-03-11 Junxian Li , Tu Lan , Haozhen Tan , Yan Meng , Haojin Zhu

The rapid advancement of Large Language Model (LLM)-driven multi-agent systems has significantly streamlined software developing tasks, enabling users with little technical expertise to develop executable applications. While these systems…

Cryptography and Security · Computer Science 2025-11-25 Xiaoqing Wang , Keman Huang , Bin Liang , Hongyu Li , Xiaoyong Du

Black-box adversarial attacks on Large Vision-Language Models (LVLMs) are challenging due to missing gradients and complex multimodal boundaries. While prior state-of-the-art transfer-based approaches like M-Attack perform well using local…

Machine Learning · Computer Science 2026-02-20 Xiaohan Zhao , Zhaoyi Li , Yaxin Luo , Jiacheng Cui , Zhiqiang Shen

Robustness of huge Transformer-based models for natural language processing is an important issue due to their capabilities and wide adoption. One way to understand and improve robustness of these models is an exploration of an adversarial…

Training Vision-Language Models (VLMs) for Graphical User Interfaces (GUI) agents via Reinforcement Learning (RL) faces critical challenges: environment-based RL requires costly interactions, while environment-free methods struggle with…

Machine Learning · Computer Science 2025-02-27 Jiani Zheng , Lu Wang , Fangkai Yang , Chaoyun Zhang , Lingrui Mei , Wenjie Yin , Qingwei Lin , Dongmei Zhang , Saravan Rajmohan , Qi Zhang

This study evaluates the resilience of large language models (LLMs) against adversarial attacks, specifically focusing on Flan-T5, BERT, and RoBERTa-Base. Using systematically designed adversarial tests through TextFooler and BERTAttack, we…

Cryptography and Security · Computer Science 2025-09-15 Taniya Gidatkar , Oluwaseun Ajao , Matthew Shardlow

Vision-Language Models (VLMs) are rapidly replacing unimodal encoders in modern retrieval and recommendation systems. While their capabilities are well-documented, their robustness against adversarial manipulation in competitive ranking…

Computation and Language · Computer Science 2026-01-21 Yixuan Du , Chenxiao Yu , Haoyan Xu , Ziyi Wang , Yue Zhao , Xiyang Hu

Recently, autonomous agents built on large language models (LLMs) have experienced significant development and are being deployed in real-world applications. These agents can extend the base LLM's capabilities in multiple ways. For example,…

Cryptography and Security · Computer Science 2024-07-31 Boyang Zhang , Yicong Tan , Yun Shen , Ahmed Salem , Michael Backes , Savvas Zannettou , Yang Zhang

Vision-Language Models (VLMs) are increasingly deployed in public sector missions, necessitating robust evaluation of their safety and vulnerability to adversarial attacks. This paper introduces a novel framework to quantify adversarial…

Computers and Society · Computer Science 2025-02-26 Maisha Binte Rashid , Pablo Rivas

Autonomous agentic AI systems powered by vision-language models (VLMs) are rapidly advancing toward real-world deployment, yet their cross-modal reasoning capabilities introduce new attack surfaces for adversarial manipulation that exploit…

Artificial Intelligence · Computer Science 2025-11-25 Hangoo Kang , Jehyeok Yeon , Gagandeep Singh

Deep reinforcement learning has shown promising results in learning control policies for complex sequential decision-making tasks. However, these neural network-based policies are known to be vulnerable to adversarial examples. This…

Computer Vision and Pattern Recognition · Computer Science 2017-10-04 Yen-Chen Lin , Ming-Yu Liu , Min Sun , Jia-Bin Huang

Large Language Models (LLMs) are increasingly used in agentic systems, where their interactions with diverse tools and environments create complex, multi-stage safety challenges. However, existing benchmarks mostly rely on static,…

Cryptography and Security · Computer Science 2026-02-03 Liming Lu , Xiang Gu , Junyu Huang , Jiawei Du , Xu Zheng , Yunhuai Liu , Yongbin Zhou , Shuchao Pang

Efficient agentic systems should incur expensive frontier-model costs only on decisions where a cheaper local model is likely to fail. Existing LLM cascades usually route whole queries before execution, but task difficulty shifts…

This report provides an architecture-led analysis of two modern vision-language models (VLMs), Qwen2.5-VL-7B-Instruct and Llama-4-Scout-17B-16E-Instruct, and explains how their architectural properties map to a practical video-to-artifact…

Computer Vision and Pattern Recognition · Computer Science 2025-12-30 Thomson Tong , Diba Darooneh

Large language models (LLMs) have significantly influenced various industries but suffer from a critical flaw, the potential sensitivity of generating harmful content, which poses severe societal risks. We developed and tested novel attack…

Computation and Language · Computer Science 2025-02-25 Yuyi Huang , Runzhe Zhan , Derek F. Wong , Lidia S. Chao , Ailin Tao

Vision-Language Models (VLMs) are increasingly used as perceptual modules for visual content reasoning, including through captioning and DeepFake detection. In this work, we expose a critical vulnerability of VLMs when exposed to subtle,…

Computer Vision and Pattern Recognition · Computer Science 2025-08-14 Jordan Vice , Naveed Akhtar , Yansong Gao , Richard Hartley , Ajmal Mian

Vision--Language Models (VLMs) show significant promise for Medical Visual Question Answering (VQA), yet their deployment in clinical settings is hindered by severe vulnerability to adversarial attacks. Standard adversarial training, while…

Artificial Intelligence · Computer Science 2025-12-23 A. A. Gde Yogi Pramana , Jason Ray , Anthony Jaya , Michael Wijaya

The objective of this study is to design and implement a reinforcement learning (RL) environment using D\&D 5E combat scenarios to challenge smaller RL agents through interaction with a robust adversarial agent controlled by advanced Large…

Artificial Intelligence · Computer Science 2025-03-21 Joseph Emmanuel DL Dayo , Michel Onasis S. Ogbinar , Prospero C. Naval