English
Related papers

Related papers: Effective, Platform-Independent GUI Testing via Im…

200 papers

Autonomous graphical user interface (GUI) agents rely on accurate GUI grounding, which maps language instructions to on-screen coordinates, to execute user commands. However, current models, whether trained via supervised fine-tuning (SFT)…

Computer Vision and Pattern Recognition · Computer Science 2026-05-28 Shaojie Zhang , Pei Fu , Ruoceng Zhang , Jiahui Yang , Anan Du , Xiuwen Xi , Shaokang Wang , Ying Huang , Bin Qin , Zhenbo Luo , Jian Luan

Exploratory GUI testing is a particularly demanding setting for MLLM agents: without predefined test scripts, an agent must autonomously navigate an application and discover defects through its own interaction. However, current evaluation…

Software Engineering · Computer Science 2026-05-29 Xiaoyi Chen , Yifei Gao , Yang Xu , Xingxing Song , Yi Zhang , Jitao Sang

The process of developing a mobile application typically starts with the ideation and conceptualization of its user interface. This concept is then translated into a set of mock-ups to help determine how well the user interface embodies the…

Software Engineering · Computer Science 2019-01-07 Carlos Bernal-Cardenas , Kevin Moran , Michele Tufano , Zichang Liu , Linyong Nan , Zhehan Shi , Denys Poshyvanyk

Graphical User Interface (GUI) agents have made substantial strides in understanding and executing user instructions across diverse platforms. Yet, grounding these instructions to precise interface elements remains challenging, especially…

Artificial Intelligence · Computer Science 2025-05-27 Xinbin Yuan , Jian Zhang , Kaixin Li , Zhuoxuan Cai , Lujian Yao , Jie Chen , Enguang Wang , Qibin Hou , Jinwei Chen , Peng-Tao Jiang , Bo Li

Background: End-user satisfaction is not only dependent on the correct functioning of the software systems but is also heavily dependent on how well those functions are performed. Therefore, performance testing plays a critical role in…

Performance · Computer Science 2020-07-24 Golrokh Hamidi

REST (Representational State Transfer) APIs have become integral for data communication and exchange due to their simplicity, scalability, and compatibility with web standards. However, ensuring REST APIs' reliability through rigorous…

Software Engineering · Computer Science 2024-10-22 Tien-Quang Nguyen , Nghia-Hieu Cong , Ngoc-Minh Quach , Hieu Dinh Vo , Son Nguyen

The recent DeepSeek-R1 has showcased the emergence of reasoning capabilities in LLMs through reinforcement learning (RL) with rule-based rewards. Despite its success in language models, its application in multi-modal domains, particularly…

Artificial Intelligence · Computer Science 2025-05-27 Zhengxi Lu , Yuxiang Chai , Yaxuan Guo , Xi Yin , Liang Liu , Hao Wang , Han Xiao , Shuai Ren , Guanjing Xiong , Hongsheng Li

Smartphones have significantly enhanced our daily learning, communication, and entertainment, becoming an essential component of modern life. However, certain populations, including the elderly and individuals with disabilities, encounter…

Robotics · Computer Science 2024-09-17 Kelin Fu , Yang Tian , Kaigui Bian

GUI is a bridge connecting user and application. Existing GUI testing tasks can be categorized into two groups: functionality testing and compatibility testing. While the functionality testing focuses on detecting application runtime bugs,…

Software Engineering · Computer Science 2022-12-29 Jiaming Ye , Mulong Xie , Siyuan Chen , Fuyuan Zhang , Lei Ma , Zhenchang Xing , Jianjun Zhao

In the current landscape of pervasive smartphones and tablets, apps frequently exist across both platforms. Although apps share most graphic user interfaces (GUIs) and functionalities across phones and tablets, developers often rebuild from…

Software Engineering · Computer Science 2023-11-07 Han Hu , Haolan Zhan , Yujin Huang , Di Liu

Testing is the most direct and effective technique to ensure software quality. Test scripts always play a more important role in mobile app testing than test cases for source code, due to the GUI-intensive and event-driven characteristics…

Software Engineering · Computer Science 2025-03-04 Shengcheng Yu , Chunrong Fang , Jia Liu , Zhenyu Chen

The state space of Android apps is huge and its thorough exploration during testing remains a major challenge. In fact, the best exploration strategy is highly dependent on the features of the app under test. Reinforcement Learning (RL) is…

Software Engineering · Computer Science 2021-12-13 Andrea Romdhana , Alessio Merlo , Mariano Ceccato , Paolo Tonella

Mobile app GUI (Graphical User Interface) pages now contain rich visual information, with the visual semantics of each page helping users understand the application logic. However, these complex visual and functional logic present new…

Software Engineering · Computer Science 2024-12-05 Zhe Liu , Cheng Li , Chunyang Chen , Junjie Wang , Mengzhuo Chen , Boyu Wu , Yawen Wang , Jun Hu , Qing Wang

Existing efforts in building Graphical User Interface (GUI) agents largely rely on the training paradigm of supervised fine-tuning on Large Vision-Language Models (LVLMs). However, this approach not only demands extensive amounts of…

Computer Vision and Pattern Recognition · Computer Science 2025-10-02 Run Luo , Lu Wang , Wanwei He , Longze Chen , Jiaming Li , Xiaobo Xia

Automated GUI testing is widely used to help ensure the quality of mobile apps. However, many GUIs require appropriate text inputs to proceed to the next page which remains a prominent obstacle for testing coverage. Considering the…

Software Engineering · Computer Science 2022-12-12 Zhe Liu , Chunyang Chen , Junjie Wang , Xing Che , Yuekai Huang , Jun Hu , Qing Wang

Mobile apps are indispensable for people's daily life, and automated GUI (Graphical User Interface) testing is widely used for app quality assurance. There is a growing interest in using learning-based techniques for automated GUI testing…

Software Engineering · Computer Science 2023-05-17 Zhe Liu , Chunyang Chen , Junjie Wang , Mengzhuo Chen , Boyu Wu , Xing Che , Dandan Wang , Qing Wang

With the popularity of smartphones and tablets, users have become accustomed to using different devices for different tasks, such as using their phones to play games and tablets to watch movies. To conquer the market, one app is often…

Human-Computer Interaction · Computer Science 2023-11-07 Han Hu , Haolan Zhan , Yujin Huang , Di Liu

The importance of mobile application (app) quality insurance is increasing with the rapid development of the mobile Internet. Automated test generation approaches, as a dominant direction of app quality insurance, follow specific models or…

Software Engineering · Computer Science 2024-06-13 Shengcheng Yu , Chunrong Fang , Mingzhe Du , Zimin Ding , Zhenyu Chen , Zhendong Su

Mobile apps provide new opportunities to people with disabilities to act independently in the world. Motivated by this trend, researchers have conducted empirical studies by using the inaccessibility issue rate of each page (i.e., screen…

Software Engineering · Computer Science 2022-03-15 Sen Chen , Chunyang Chen , Lingling Fan , Mingming Fan , Xian Zhan , Yang Liu

Background. Evidence suggests that mobile applications are not thoroughly tested as their desktop counterparts. In particular GUI testing is generally limited. Like web-based applications, mobile apps suffer from GUI test fragility, i.e.…

Software Engineering · Computer Science 2017-11-13 Riccardo Coppola , Maurizio Morisio , Marco Torchiano