English
Related papers

Related papers: Vision-Based Mobile App GUI Testing: A Survey

200 papers

Graphical User Interface (GUI) Agents have emerged as a transformative paradigm in human-computer interaction, evolving from rule-based automation scripts to sophisticated AI-driven systems capable of understanding and executing complex…

Human-Computer Interaction · Computer Science 2025-06-05 Fei Tang , Haolei Xu , Hang Zhang , Siqi Chen , Xingyu Wu , Yongliang Shen , Wenqi Zhang , Guiyang Hou , Zeqi Tan , Yuchen Yan , Kaitao Song , Jian Shao , Weiming Lu , Jun Xiao , Yueting Zhuang

Mobile app development involves a unique set of challenges including device fragmentation and rapidly evolving platforms, making testing a difficult task. The design space for a comprehensive mobile testing strategy includes features,…

Software Engineering · Computer Science 2018-01-22 Mario Linares Vasquez , Kevin Moran , Denys Poshyvanyk

Autonomous agents that navigate Graphical User Interfaces (GUIs) to automate tasks like document editing and file management can greatly enhance computer workflows. While existing research focuses on online settings, desktop environments,…

Nowadays, mobile applications are essential tools for everyday life, providing users with anytime, anywhere access to up-to-date information, communication, and entertainment. Needless to say, hardware limitations and the diverse needs of…

Human-Computer Interaction · Computer Science 2025-09-29 Pawel Weichbroth , Tomasz Szot

Existing online benchmarks for mobile GUI agents remain largely app-centric and task-homogeneous, failing to reflect the diversity and instability of real-world mobile usage. To this end, we introduce VenusBench-Mobile, a challenging online…

Human-Computer Interaction · Computer Science 2026-04-09 Yichen Gong , Zhuohan Cai , Sunhao Dai , Yuqi Zhou , Zhangxuan Gu , Changhua Meng , Shuheng Shen

Graphical user interfaces (GUIs) are integral parts of software systems that require interactions from their users. Software testers have paid special attention to GUI testing in the last decade, and have devised techniques that are…

Software Engineering · Computer Science 2017-03-29 Valéria Lelli , Arnaud Blouin , Benoit Baudry

As mobile applications (apps) become ubiquitous in everyday life, it is crucial for developers to prioritize accessibility for users with diverse abilities. While previous research has identified widespread accessibility issues and raised…

Software Engineering · Computer Science 2026-01-21 Amila Indika , Rick Kazman , Anthony Peruma

The fragmentation problem has extended from Android to different platforms, such as iOS, mobile web, and even mini-programs within some applications (app). In such a situation, recording and replaying test scripts is a popular automated…

Software Engineering · Computer Science 2021-02-23 Shengcheng Yu , Chunrong Fang , Yexiao Yun , Yang Feng

Software testing is a fundamental process of software development, and prior work has shown that visualizations of test results support testers' decision-making. However, Human-Computer Interaction research on software testing has yet to…

Human-Computer Interaction · Computer Science 2026-05-07 Brandon Lit , Anthony Maocheia-Ricci , Thomas Driscoll

Utilizing Graphic User Interface (GUI) for human-computer interaction is essential for accessing a wide range of digital tools. Recent advancements in Vision Language Models (VLMs) highlight the compelling potential to develop versatile…

Artificial Intelligence · Computer Science 2025-06-02 Wentong Chen , Junbo Cui , Jinyi Hu , Yujia Qin , Junjie Fang , Yue Zhao , Chongyi Wang , Jun Liu , Guirong Chen , Yupeng Huo , Yuan Yao , Yankai Lin , Zhiyuan Liu , Maosong Sun

Intense competition in the mobile apps market means it is important to maintain high levels of app reliability to avoid losing users. Yet despite its importance, app reliability is underexplored in the research literature. To address this…

Software Engineering · Computer Science 2022-06-22 Chathrie Wimalasooriya , Sherlock A. Licorish , Daniel Alencar da Costa , Stephen G. MacDonell

Visual language models (VLMs) empower mobile GUI agents to interpret complex mobile screens and respond to user requests. Training such capable agents requires large-scale, high-quality mobile GUI data. However, existing mobile GUI datasets…

Human-Computer Interaction · Computer Science 2025-11-26 Longxi Gao , Li Zhang , Shihe Wang , Pengzhi Gao , Wei Liu , Jian Luan , Shangguang Wang , Yuanchun Li , Mengwei Xu

Smartphones and tablets have established themselves as mainstays in the modern computing landscape. It is conceivable that in the near future such devices may supplant laptops and desktops, becoming many users primary means of carrying out…

Software Engineering · Computer Science 2018-01-19 Kevin Moran

Mobile applications have become an essential part of our daily lives, making ensuring their quality an important activity. Graphical User Interface (GUI) testing is a quality assurance method that has frequently been used for mobile apps.…

Software Engineering · Computer Science 2025-09-11 Chenhui Cui , Tao Li , Junjie Wang , Chunyang Chen , Dave Towey , Rubing Huang

Mobile apps are indispensable for people's daily life. Complementing with automated GUI testing, manual testing is the last line of defence for app quality. However, the repeated actions and easily missing of functionalities make manual…

Software Engineering · Computer Science 2022-01-31 Zhe Liu , Chunyang Chen , Junjie Wang , Yuekai Huang , Jun Hu , Qing Wang

Mobile apps provide new opportunities to people with disabilities to act independently in the world. Motivated by this trend, researchers have conducted empirical studies by using the inaccessibility issue rate of each page (i.e., screen…

Software Engineering · Computer Science 2022-03-15 Sen Chen , Chunyang Chen , Lingling Fan , Mingming Fan , Xian Zhan , Yang Liu

Context: Accessibility issues (e.g., small size and narrow interval) in mobile applications (apps) lead to obstacles for billions of low vision users in interacting with Graphical User Interfaces (GUIs). Although GUI accessibility scanning…

Software Engineering · Computer Science 2025-02-21 Mengxi Zhang , Huaxiao Liu , Shenning Song , Chunyang Chen , Pei Huang , Jian Zhao

As mobile application (app) functionalities grow increasingly complex and their iterations accelerate, ensuring high reliability presents significant challenges. While functionality-oriented GUI testing has attracted growing research…

Software Engineering · Computer Science 2026-02-16 Jinlong He , Changwei Xia , Binru Huang , Jiwei Yan , Jun Yan , Jian Zhang

Mobile applications (apps) have become an essential part of everyday life, offering convenient access to services such as banking, healthcare, and shopping. With these apps handling sensitive personal and financial data, ensuring their…

Cryptography and Security · Computer Science 2024-08-20 Anthony Peruma , Timothy Huo , Ana Catarina Araújo , Jake Imanaka , Rick Kazman

Searching for relative mobile user interface (UI) design examples can aid interface designers in gaining inspiration and comparing design alternatives. However, finding such design examples is challenging, especially as current search…

Human-Computer Interaction · Computer Science 2021-02-11 Sara Bunian , Kai Li , Chaima Jemmali , Casper Harteveld , Yun Fu , Magy Seif El-Nasr