English
Related papers

Related papers: Multimodal Safety Evaluation in Generative Agent S…

200 papers

Recent advances in vision-language models have enabled mobile GUI agents to perceive visual interfaces and execute user instructions, but reliable prediction of action consequences remains critical for long-horizon and high-risk…

Artificial Intelligence · Computer Science 2026-05-25 Weikai Xu , Kun Huang , Yunren Feng , Jiaxing Li , Yuhan Chen , Yuxuan Liu , Zhizheng Jiang , Heng Qu , Pengzhi Gao , Wei Liu , Jian Luan , Xiaolin Hu , Bo An

Designing and evaluating personalized and proactive assistant agents remains challenging due to the time, cost, and ethical concerns associated with human-in-the-loop experimentation. Existing Human-Computer Interaction (HCI) methods often…

Human-Computer Interaction · Computer Science 2025-11-25 Ziyi Xuan , Yiwen Wu , Xuhai Xu , Vinod Namboodiri , Mooi Choo Chuah , Yu Yang

A multimodal AI agent is characterized by its ability to process and learn from various types of data, including natural language, visual, and audio inputs, to inform its actions. Despite advancements in large language models that…

Computation and Language · Computer Science 2024-04-19 Wei Chen , Zhiyuan Li

There are growing concerns about the risks posed by AI companion applications designed for emotional engagement. Existing safety evaluations often rely on self-reported user data or interviews, offering limited insights into real-time…

Computation and Language · Computer Science 2026-05-04 Prerna Juneja , Lika Lomidze

As language models evolve into autonomous agents that act and communicate on behalf of users, ensuring safety in multi-agent ecosystems becomes a central challenge. Interactions between personal assistants and external service providers…

Cryptography and Security · Computer Science 2025-11-10 Amr Gomaa , Ahmed Salem , Sahar Abdelnabi

Simulation environments are good for learning different driving tasks like lane changing, parking or handling intersections etc. in an abstract manner. However, these simulation environments often restrict themselves to operate under…

Machine Learning · Computer Science 2021-11-01 Ashish Rana , Avleen Malhi

Believable proxies of human behavior can empower interactive applications ranging from immersive environments to rehearsal spaces for interpersonal communication to prototyping tools. In this paper, we introduce generative…

Human-Computer Interaction · Computer Science 2023-08-08 Joon Sung Park , Joseph C. O'Brien , Carrie J. Cai , Meredith Ringel Morris , Percy Liang , Michael S. Bernstein

Generative AI systems produce a range of risks. To ensure the safety of generative AI systems, these risks must be evaluated. In this paper, we make two main contributions toward establishing such evaluations. First, we propose a…

Experience-driven self-evolution has emerged as a promising paradigm for improving the autonomy of large language model agents, yet its reliance on self-curated experience introduces underexplored safety risks. In this study, we investigate…

Computation and Language · Computer Science 2026-04-21 Weixiang Zhao , Yichen Zhang , Yingshuo Wang , Yang Deng , Yanyan Zhao , Xuda Zhi , Yongbo Huang , HaoHe , Wanxiang Che , Bing Qin , Ting Liu

Multiagent social network simulations are an avenue that can bridge the communication gap between the public and private platforms in order to develop solutions to a complex array of issues relating to online safety. While there are…

Social and Information Networks · Computer Science 2023-11-28 Aditya Surve , Archit Rathod , Mokshit Surana , Gautam Malpani , Aneesh Shamraj , Sainath Reddy Sankepally , Raghav Jain , Swapneel S Mehta

Agentic AI systems -- Large Language Models (LLMs) augmented with planning, tool use, memory, and long-horizon interactions -- can execute complex tasks autonomously, but their multi-step trajectories introduce new failure modes that…

Artificial Intelligence · Computer Science 2026-05-26 Jinhu Qi , Muzhi Li , Jiahong Liu , Yuqin Shu , Dianzhi Yu , Shicheng Ma , Wenqian Cui , Yiyang Zhao , Yiyi Chen , Ruoxi Jiang , Irwin King , Zenglin Xu

Large Language Models (LLMs) exhibit substantial promise in enhancing task-planning capabilities within embodied agents due to their advanced reasoning and comprehension. However, the systemic safety of these agents remains an underexplored…

Artificial Intelligence · Computer Science 2025-04-22 Yuting Huang , Leilei Ding , Zhipeng Tang , Tianfu Wang , Xinrui Lin , Wuyang Zhang , Mingxiao Ma , Yanyong Zhang

Despite the growing use of large language models (LLMs) in clinical dialogue systems, existing evaluations focus on task completion or fluency, offering little insight into the behavioral and risk management requirements essential for…

Large Language Models (LLMs) have revolutionized the simulation of agent societies, enabling autonomous planning, memory formation, and social interactions. However, existing frameworks often overlook systematic evaluations for event…

Artificial Intelligence · Computer Science 2025-09-17 Yuyang Tian , Shunqiang Mao , Wenchang Gao , Lanlan Qiu , Tianxing He

Extended interaction with large language models (LLMs) has been linked to the reinforcement of delusional beliefs, a phenomenon attracting growing clinical and public concern. Yet most empirical work evaluates model safety in brief…

Human-Computer Interaction · Computer Science 2026-04-24 Luke Nicholls , Robert Hutto , Zephrah Soto , Hamilton Morrin , Thomas Pollak , Raj Korpan , Cheryl Carmichael

The increasing adoption of Reinforcement Learning in safety-critical systems domains such as autonomous vehicles, health, and aviation raises the need for ensuring their safety. Existing safety mechanisms such as adversarial training,…

Machine Learning · Computer Science 2021-11-11 Paulina Stevia Nouwou Mindom , Amin Nikanjam , Foutse Khomh , John Mullins

With the wide application of multimodal foundation models in intelligent agent systems, scenarios such as mobile device control, intelligent assistant interaction, and multimodal task execution are gradually relying on such large…

Artificial Intelligence · Computer Science 2025-07-02 Siyuan Liang , Tianmeng Fang , Zhe Liu , Aishan Liu , Yan Xiao , Jinyuan He , Ee-Chien Chang , Xiaochun Cao

Language model (LM) agents have demonstrated significant potential for automating real-world tasks, yet they pose a diverse array of potential, severe risks in safety-critical scenarios. In this work, we identify a significant gap between…

Artificial Intelligence · Computer Science 2025-08-20 Yuzhi Tang , Tianxiao Li , Elizabeth Li , Chris J. Maddison , Honghua Dong , Yangjun Ruan

Multimodal large language models (MLLMs) enable interaction over both text and images, but their safety behavior can be driven by unimodal shortcuts instead of true joint intent understanding. We introduce CSR-Bench, a benchmark for…

Artificial Intelligence · Computer Science 2026-02-04 Yuxuan Liu , Yuntian Shi , Kun Wang , Haoting Shen , Kun Yang

Large Language Models demonstrate strong reasoning and generation abilities, yet their behavior in multi-turn tasks often lacks reliability and verifiability. We present a task completion framework that enables LLM-based agents to act under…

Artificial Intelligence · Computer Science 2025-12-15 Gonca Gürsun