English
Related papers

Related papers: Manipulating Multimodal Agents via Cross-Modal Pro…

200 papers

Large Vision-Language Models (LVLMs) have transformed multi-modal understanding, excelling in tasks like image captioning and visual question answering by integrating visual and textual inputs. However, their robustness against adversarial…

Computer Vision and Pattern Recognition · Computer Science 2026-05-27 Xiang Fang , Wanlong Fang , Changshuo Wang

Large Language Model (LLM) agents are increasingly being deployed as conversational assistants capable of performing complex real-world tasks through tool integration. This enhanced ability to interact with external systems and process…

Cryptography and Security · Computer Science 2024-12-24 Feiran Jia , Tong Wu , Xin Qin , Anna Squicciarini

Multimodal Large Language Models (MLLMs) demonstrate exceptional performance in cross-modality interaction, yet they also suffer adversarial vulnerabilities. In particular, the transferability of adversarial examples remains an ongoing…

Computer Vision and Pattern Recognition · Computer Science 2025-07-22 Hao Cheng , Erjia Xiao , Jiayan Yang , Jinhao Duan , Yichi Wang , Jiahang Cao , Qiang Zhang , Le Yang , Kaidi Xu , Jindong Gu , Renjing Xu

Large Language Models (LLMs) have evolved into Multimodal Large Language Models (MLLMs), significantly enhancing their capabilities by integrating visual information and other types, thus aligning more closely with the nature of human…

Cryptography and Security · Computer Science 2025-06-03 Youze Wang , Wenbo Hu , Yinpeng Dong , Jing Liu , Hanwang Zhang , Richang Hong

Large Language Models (LLMs) are increasingly equipped with capabilities of real-time web search and integrated with protocols like Model Context Protocol (MCP). This extension could introduce new security vulnerabilities. We present a…

Cryptography and Security · Computer Science 2025-05-23 Junjie Xiong , Changjia Zhu , Shuhang Lin , Chong Zhang , Yongfeng Zhang , Yao Liu , Lingyao Li

The integration of external data services (e.g., Model Context Protocol, MCP) has made large language model-based agents increasingly powerful for complex task execution. However, this advancement introduces critical security…

Cryptography and Security · Computer Science 2026-02-25 Che Wang , Jiaming Zhang , Ziqi Zhang , Zijie Wang , Yinghui Wang , Jianbo Gao , Tao Wei , Zhong Chen , Wei Yang Bryan Lim

Text-to-image (T2I) models have raised increasing safety concerns due to their capacity to generate NSFW and other banned objects. To mitigate these risks, safety filters and concept removal techniques have been introduced to block…

Cryptography and Security · Computer Science 2026-01-13 Xi Ye , Yiwen Liu , Lina Wang , Run Wang , Geying Yang , Yufei Hou , Jiayi Yu

Large Language Model (LLM)-based agents have demonstrated remarkable capabilities in reasoning, planning, and tool usage. The recently proposed Model Context Protocol (MCP) has emerged as a unifying framework for integrating external tools…

Cryptography and Security · Computer Science 2025-12-03 Yuanhe Zhang , Weiliu Wang , Zhenhong Zhou , Kun Wang , Jie Zhang , Li Sun , Yang Liu , Sen Su

Embodied agents exhibit immense potential across a multitude of domains, making the assurance of their behavioral safety a fundamental prerequisite for their widespread deployment. However, existing research predominantly concentrates on…

Artificial Intelligence · Computer Science 2025-06-23 Ning Wang , Zihan Yan , Weiyang Li , Chuan Ma , He Chen , Tao Xiang

Large Language Model (LLM) agents can leverage tools such as Google Search to complete complex tasks. However, this tool usage introduces the risk of indirect prompt injections, where malicious instructions hidden in tool outputs can…

Machine Learning · Computer Science 2025-10-08 Zizhao Wang , Dingcheng Li , Vaishakh Keshava , Phillip Wallis , Ananth Balashankar , Peter Stone , Lukas Rutishauser

The proliferation of open-weight Large Language Models (LLMs) has democratized agentic AI, yet fine-tuned weights are frequently shared and adopted with limited scrutiny beyond leaderboard performance. This creates a risk where third-party…

Cryptography and Security · Computer Science 2026-03-05 Bhanu Pallakonda , Mikkel Hindsbo , Sina Ehsani , Prag Mishra

Large language models (LLMs) are becoming a popular tool as they have significantly advanced in their capability to tackle a wide range of language-based tasks. However, LLMs applications are highly vulnerable to prompt injection attacks,…

Computation and Language · Computer Science 2024-11-11 Md Abdur Rahman , Fan Wu , Alfredo Cuzzocrea , Sheikh Iqbal Ahamed

We introduce a multicrossmodal LLM-agent framework motivated by the growing volume and diversity of materials-science data ranging from high-resolution microscopy and dynamic simulation videos to tabular experiment logs and sprawling…

Materials Science · Physics 2025-05-22 Adib Bazgir , Rama chandra Praneeth Madugula , Yuwen Zhang

The rapid progress of Multi-Modal Large Language Models (MLLMs) has significantly advanced downstream applications. However, this progress also exposes serious transferable adversarial vulnerabilities. In general, existing adversarial…

Computer Vision and Pattern Recognition · Computer Science 2026-03-24 Yuanbo Li , Tianyang Xu , Cong Hu , Tao Zhou , Xiao-Jun Wu , Josef Kittler

Prompt injection attacks pose a critical threat to large language models (LLMs), enabling goal hijacking and data leakage. Prompt guard models, though effective in defense, suffer from over-defense -- falsely flagging benign inputs as…

Computation and Language · Computer Science 2025-04-01 Hao Li , Xiaogeng Liu

Agentic AI coding editors driven by large language models have recently become more popular due to their ability to improve developer productivity during software development. Modern editors such as Cursor are designed not just for code…

Cryptography and Security · Computer Science 2026-04-29 Yue Liu , Yanjie Zhao , Yunbo Lyu , Ting Zhang , Haoyu Wang , David Lo

Multimodal web agents that process both screenshots and accessibility trees are increasingly deployed to interact with web interfaces, yet their dual-stream architecture opens an underexplored attack surface: an adversary who injects…

Machine Learning · Computer Science 2026-03-05 Haoyu Liu , Dingcheng Li , Lukas Rutishauser , Zeyu Zheng

Recently, autonomous agents built on large language models (LLMs) have experienced significant development and are being deployed in real-world applications. These agents can extend the base LLM's capabilities in multiple ways. For example,…

Cryptography and Security · Computer Science 2024-07-31 Boyang Zhang , Yicong Tan , Yun Shen , Ahmed Salem , Michael Backes , Savvas Zannettou , Yang Zhang

Vision Large Language Models (VLLMs) integrate visual data processing, expanding their real-world applications, but also increasing the risk of generating unsafe responses. In response, leading companies have implemented Multi-Layered…

Computer Vision and Pattern Recognition · Computer Science 2025-02-11 Yijun Yang , Lichao Wang , Xiao Yang , Lanqing Hong , Jun Zhu

While vision-language pre-training model (VLP) has shown revolutionary improvements on various vision-language (V+L) tasks, the studies regarding its adversarial robustness remain largely unexplored. This paper studied the adversarial…

Machine Learning · Computer Science 2022-10-21 Jiaming Zhang , Qi Yi , Jitao Sang