English
Related papers

Related papers: On the Robustness of GUI Grounding Models Against …

200 papers

Robustness of machine learning models is critical for security related applications, where real-world adversaries are uniquely focused on evading neural network based detectors. Prior work mainly focus on crafting adversarial examples (AEs)…

Machine Learning · Computer Science 2021-11-01 Ecenaz Erdemir , Jeffrey Bickford , Luca Melis , Sergul Aydore

The rapid advancement of Generative Artificial Intelligence (GenAI) capabilities is accompanied by a concerning rise in its misuse. In particular the generation of credible misinformation in the form of images poses a significant threat to…

Computer Vision and Pattern Recognition · Computer Science 2025-06-04 Sina Mavali , Jonas Ricker , David Pape , Asja Fischer , Lea Schönherr

Graph neural networks (GNNs) learn node representations by passing and aggregating messages between neighboring nodes. GNNs have been applied successfully in several application domains and achieved promising performance. However, GNNs…

Machine Learning · Computer Science 2021-12-14 Zeyu Zhang , Yulong Pei

Graphical user interface (GUI) agents autonomously complete tasks across platforms (\eg, Linux) by sequentially decomposing user instructions into action proposals that iteratively interact with visual elements in the evolving environment.…

A cursory reading of the literature suggests that we have made a lot of progress in designing effective adversarial defenses for Graph Neural Networks (GNNs). Yet, the standard methodology has a serious flaw - virtually all of the defenses…

Machine Learning · Computer Science 2023-02-01 Felix Mujkanovic , Simon Geisler , Stephan Günnemann , Aleksandar Bojchevski

Recent work has demonstrated that deep neural networks are vulnerable to adversarial examples---inputs that are almost indistinguishable from natural data and yet classified incorrectly by the network. In fact, some of the latest findings…

Machine Learning · Statistics 2019-09-06 Aleksander Madry , Aleksandar Makelov , Ludwig Schmidt , Dimitris Tsipras , Adrian Vladu

Graphical user interface (GUI) grounding is a key capability for computer-use agents, mapping natural-language instructions to actionable regions on the screen. Existing Multimodal Large Language Model (MLLM) approaches typically formulate…

Computer Vision and Pattern Recognition · Computer Science 2026-03-30 Shijie Zhou , Viet Dac Lai , Hao Tan , Jihyung Kil , Wanrong Zhu , Changyou Chen , Ruiyi Zhang

Understanding the spatial arrangement and nature of real-world objects is of paramount importance to many complex engineering tasks, including autonomous navigation. Deep learning has revolutionized state-of-the-art performance for tasks in…

Computer Vision and Pattern Recognition · Computer Science 2019-04-02 Matthew Wicker , Marta Kwiatkowska

Despite their unmatched performance, deep neural networks remain susceptible to targeted attacks by nearly imperceptible levels of adversarial noise. While the underlying cause of this sensitivity is not well understood, theoretical…

Machine Learning · Computer Science 2020-12-01 George Cazenavette , Calvin Murdock , Simon Lucey

Recent advances in vision-language models (VLMs) and reinforcement learning (RL) have driven progress in GUI automation. However, most existing methods rely on static, one-shot visual inputs and passive perception, lacking the ability to…

Artificial Intelligence · Computer Science 2026-01-16 Chen Chen , Jiawei Shao , Dakuan Lu , Haoyi Hu , Xiangcheng Liu , Hantao Yao , Wu Liu

Graphical User Interface (GUI) has become one of the most significant parts of mobile applications (apps). It is a direct bridge between mobile apps and end users, which directly affects the end user's experience. Neglecting GUI quality can…

Software Engineering · Computer Science 2025-10-21 Shengcheng Yu , Chunrong Fang , Ziyuan Tuo , Quanjun Zhang , Chunyang Chen , Zhenyu Chen , Zhendong Su

Rapid advancements of deep learning are accelerating adoption in a wide variety of applications, including safety-critical applications such as self-driving vehicles, drones, robots, and surveillance systems. These advancements include…

Computer Vision and Pattern Recognition · Computer Science 2024-05-06 Firuz Juraev , Mohammed Abuhamad , Simon S. Woo , George K Thiruvathukal , Tamer Abuhmed

Assured AI in unrestricted settings is a critical problem. Our framework addresses AI assurance challenges lying at the intersection of domain adaptation, fairness, and counterfactuals analysis, operating via the discovery and intervention…

Machine Learning · Computer Science 2021-11-19 William Paul , Philippe Burlina

Adversarial robustness research primarily focuses on L_p perturbations, and most defenses are developed with identical training-time and test-time adversaries. However, in real-world applications developers are unlikely to have access to…

Visual robustness under real-world conditions remains a critical bottleneck for modern reinforcement learning agents. In contrast, biological systems such as mice show remarkable resilience to environmental changes, maintaining stable…

Neurons and Cognition · Quantitative Biology 2025-09-19 Marius Schneider , Joe Canzano , Jing Peng , Yuchen Hou , Spencer LaVere Smith , Michael Beyeler

Appearance-based gait recognition have achieved strong performance on controlled datasets, yet systematic evaluation of its robustness to real-world corruptions and silhouette variability remains lacking. We present RobustGait, a framework…

Computer Vision and Pattern Recognition · Computer Science 2026-02-26 Reeshoon Sayera , Akash Kumar , Sirshapan Mitra , Prudvi Kamtam , Yogesh S Rawat

Graphical user interfaces (GUIs) are integral parts of software systems that require interactions from their users. Software testers have paid special attention to GUI testing in the last decade, and have devised techniques that are…

Software Engineering · Computer Science 2017-03-29 Valéria Lelli , Arnaud Blouin , Benoit Baudry

Recent studies have revealed that GNNs are vulnerable to adversarial attacks. To defend against such attacks, robust graph structure refinement (GSR) methods aim at minimizing the effect of adversarial edges based on node features, graph…

Machine Learning · Computer Science 2024-03-05 Yeonjun In , Kanghoon Yoon , Kibum Kim , Kijung Shin , Chanyoung Park

Graph Neural Networks (GNNs) have emerged as a dominant paradigm for learning on graph-structured data, thanks to their ability to jointly exploit node features and relational information encoded in the graph topology. This joint modeling,…

Machine Learning · Computer Science 2025-12-30 Yongyu Wang

While deep neural networks (DNNs) have revolutionized many fields, their fragility to carefully designed adversarial attacks impedes the usage of DNNs in safety-critical applications. In this paper, we strive to explore the robust features…

Computer Vision and Pattern Recognition · Computer Science 2023-09-12 Hong Wang , Yuefan Deng , Shinjae Yoo , Yuewei Lin