English

How Smart Is Your GUI Agent? A Framework for the Future of Software Interaction

Software Engineering 2026-02-13 v1 Artificial Intelligence Computer Vision and Pattern Recognition Human-Computer Interaction

Abstract

GUI agents are rapidly becoming a new interaction to software, allowing people to navigate web, desktop and mobile rather than execute them click by click. Yet ``agent'' is described with radically different degrees of autonomy, obscuring capability, responsibility and risk. We call for conceptual clarity through GUI Agent Autonomy Levels (GAL), a six-level framework that makes autonomy explicit and helps benchmark progress toward trustworthy software interaction.

Keywords

Cite

@article{arxiv.2602.11514,
  title  = {How Smart Is Your GUI Agent? A Framework for the Future of Software Interaction},
  author = {Sidong Feng and Chunyang Chen},
  journal= {arXiv preprint arXiv:2602.11514},
  year   = {2026}
}
R2 v1 2026-07-01T10:32:56.251Z