English
Related papers

Related papers: Vision-Based Mobile App GUI Testing: A Survey

200 papers

GUI responsiveness is critical for a positive user experience in mobile applications. Even brief delays in visual feedback can frustrate users and lead to negative reviews. However, detecting and quantifying such user-perceived delays…

Software Engineering · Computer Science 2025-08-05 Wei Liu , Linqiang Guo , Yi Wen Heng , Chenglin Li , Tse-Hsun , Chen , Ahmed E. Hassan

With the rapid development of web technology, more and more software applications have become web-based in the past decades. To ensure software quality and user experience, various techniques have been proposed to automatically test web…

Software Engineering · Computer Science 2024-10-17 Siyi Wang , Sinan Wang , Yujia Fan , Xiaolei Li , Yepang Liu

This paper presents MagicGUI, a foundational mobile GUI agent designed to address critical challenges in perception, grounding, and reasoning within real-world mobile GUI environments. The framework is underpinned by following six key…

Multi-window mobile scenarios, such as split-screen and foldable modes, make GUI display defects more likely by forcing applications to adapt to changing window sizes and dynamic layout reflow. Existing detection techniques are limited in…

Software Engineering · Computer Science 2026-04-22 Xinyao Zhang , Rui Wang , Jinhao Cui , Haotian Huang , Wei Xue , Wenhua Hu , Jianwen Xiang , Rui Hao

Users may face challenges while designing graphical user interfaces, due to a lack of relevant experience and guidance. This paper aims to investigate the issues that users with no experience face during the design process, and how to…

Human-Computer Interaction · Computer Science 2020-01-17 Chunggi Lee , Sanghoon Kim , Dongyun Han , Hongjun Yang , Young-Woo Park , Bum Chul Kwon , Sungahn Ko

Voice Assistant (VA) in smartphones has become very popular with millions of users nowadays. A key trend is the rise of custom VA embedding, which enables users to perform the customized tasks of their favorite app through voice control.…

Software Engineering · Computer Science 2024-07-26 Garrett Weaver , Xue Qin

GUI testing is an essential quality assurance process in mobile app development. However, the creation and maintenance of GUI tests for mobile apps are resource-intensive and costly. Recognizing that many apps share similar functionalities,…

Software Engineering · Computer Science 2025-04-04 Xiaolei Li , Jialun Cao , Yepang Liu , Shing-Chi Cheung , Hailong Wang

Mobile applications (apps) are integral to our daily lives, offering diverse services and functionalities. They enable sighted users to access information coherently in an extremely convenient manner. However, it remains unclear if visually…

Software Engineering · Computer Science 2025-02-24 Mengxi Zhang , Huaxiao Liu , Yuheng Zhou , Chunyang Chen , Pei Huang , Jian Zhao

The integration of artificial intelligence (AI) into mobile applications has significantly transformed various domains, enhancing user experiences and providing personalized services through advanced machine learning (ML) and deep learning…

Software Engineering · Computer Science 2024-09-30 Yinghua Li , Xueqi Dang , Haoye Tian , Tiezhu Sun , Zhijie Wang , Lei Ma , Jacques Klein , Tegawendé F. Bissyandé

Enabling fully automated testing of mobile applications has recently become an important topic of study for both researchers and practitioners. A plethora of tools and approaches have been proposed to aid mobile developers both by…

Software Engineering · Computer Science 2018-01-22 Mario Linares Vasquez , Carlos Bernal-Cardenas , Kevin Moran , Denys Poshyvanyk

In this paper we investigate the effect of long-term GUI changes occurring during application development on the reusability of existing GUI test cases. We conduct an empirical evaluation on two complex, open-source GUI-driven applications…

Software Engineering · Computer Science 2017-02-24 Arthur-Jozsef Molnar

GUI testing checks if a software system behaves as expected when users interact with its graphical interface, e.g., testing specific functionality or validating relevant use case scenarios. Currently, deciding what to test at this high…

Software Engineering · Computer Science 2023-11-16 Juyeon Yoon , Robert Feldt , Shin Yoo

Smartphones have significantly enhanced our daily learning, communication, and entertainment, becoming an essential component of modern life. However, certain populations, including the elderly and individuals with disabilities, encounter…

Robotics · Computer Science 2024-09-17 Kelin Fu , Yang Tian , Kaigui Bian

Mobile apps are indispensable for people's daily life, and automated GUI (Graphical User Interface) testing is widely used for app quality assurance. There is a growing interest in using learning-based techniques for automated GUI testing…

Software Engineering · Computer Science 2023-05-17 Zhe Liu , Chunyang Chen , Junjie Wang , Mengzhuo Chen , Boyu Wu , Xing Che , Dandan Wang , Qing Wang

With the recent growth of conversational systems and intelligent assistants such as Apple Siri and Google Assistant, mobile devices are becoming even more pervasive in our lives. As a consequence, users are getting engaged with the mobile…

Information Retrieval · Computer Science 2018-07-16 Mohammad Aliannejadi , Hamed Zamani , Fabio Crestani , W. Bruce Croft

Teleoperation emerged as a promising fallback for situations beyond the capabilities of automated vehicles. Nevertheless, teleoperation still faces challenges, such as reduced situational awareness. Since situational awareness is primarily…

Human-Computer Interaction · Computer Science 2025-05-01 Maria-Magdalena Wolf , Henrik Schmidt , Michael Christl , Jana Fank , Frank Diermeyer

Many accessibility features available on mobile platforms require applications (apps) to provide complete and accurate metadata describing user interface (UI) components. Unfortunately, many apps do not provide sufficient metadata for…

Smartphones are now ubiquitous, yet our understanding of user interface factors that maximize mobile user experience (UX), is still limited. This work presents a controlled experiment, which investigated factors that affect the usability…

Human-Computer Interaction · Computer Science 2021-02-05 Lasse Einfeldt , Auriol Degbelo

Recent advances in Large Language Model (LLM)-based agents have shown remarkable progress in code generation. However, current agent methods mainly rely on text-output-based feedback (e.g. command-line outputs) for multi-round debugging and…

Software Engineering · Computer Science 2026-04-23 Zhilin Liu , Ye Huang , Ting Xie , Ruizhi Zhang , Wen Li , Lixin Duan

Detecting and fixing compatibility issues is critical for mobile game development. The rapid evolution of mobile operating systems and device fragmentation make it challenging for developers to timely address these issues across diverse…

Software Engineering · Computer Science 2025-04-30 Zihe Song , Yingfeng Chen , Lei Ma , Shangjie Lu , Honglei Lin , Changjie Fan , Wei Yang