Related papers: System Administrators Prefer Command Line Interfac…
We present the results of an exploratory study on how pairs interact with speech commands and touch gestures on a wall-sized display during a collaborative sensemaking task. Previous work has shown that speech commands, alone or in…
This paper examines how different types of large language model (LLM) agents perform on scientific visualization (SciVis) tasks, where users generate visualization workflows from natural-language instructions. We compare three primary…
Large Language Model (LLM)-powered web GUI agents are increasingly automating everyday online tasks. Despite their popularity, little is known about how users' preferences and values impact agents' reasoning and behavior. In this work, we…
Based on the fundamental constraints of natural way of interacting such as speech, touch, contextual and environmental awareness,immersive 3D experiences-all with a goal of a computer that can see listen, learn talk and act. We drive a set…
Utilizing Large Language Models (LLMs) for complex tasks is challenging, often involving a time-consuming and uncontrollable prompt engineering process. This paper introduces a novel human-LLM interaction framework, Low-code LLM. It…
Large language models (LLMs) are increasingly seen as assistants, copilots, and consultants, capable of supporting a wide range of tasks through natural conversation. However, most systems remain constrained by a linear request-response…
Highly customized graphical user interfaces play a major role in today's software applications. There exist many technologies that support developing them. But these technologies demand writing a significant amount of unintuitive code to…
Interfaces are recognized as an important mechanism to define contracts governing interactions between semi-independent software modules. Well-designed interfaces significantly reduce software complexity and ease maintainability by…
In this paper, we define visual log of a software system as data capturing the interactions between its users and its graphic user interface (GUI), such as screen-shots and screen recordings. We vision that mining such visual log could be…
Users can't talk with computers in their natural language (machine codes), so there are interfaces that allow such communication. 40 years ago the outcome of computer programs was in the form of long listings covered by numbers and even the…
AI agents are vulnerable to prompt injection attacks, where malicious content hijacks agent behavior to steal credentials or cause financial loss. The only known robust defense is architectural isolation that strictly separates trusted task…
Toolbar is one among the popular elements of a graphical user interface. The other popular elements of graphical user interface are buttons, menu, scrollbar, dialog box etc., all of which provide easy access to various functions of a GUI…
Color is an important aspect of any graphical user interface (GUI). Color is used to make a GUI attractive and meaningful. But there are difficulties in using colors too. Improper use of color can result in adverse effects. Wrong colors at…
GUI agents drive applications through their visual interfaces instead of programmatic APIs, interacting with arbitrary software via taps, swipes, and keystrokes, reaching a long tail of applications that CLI-based agents cannot. Yet…
In the past decades, integrated development environments (IDEs) have been largely advanced to facilitate common software engineering tasks. Yet, with growing information needs driven by increasing complexity in developing modern…
Protecting sensitive data is an essential part of security in cloud computing. However, only specific privileged individuals have access to view or interact with this data; therefore, it is unscalable to depend on these individuals also to…
As vehicle automation technology continues to mature, there is a necessity for robust remote monitoring and intervention features. These are essential for intervening during vehicle malfunctions, challenging road conditions, or in areas…
Product Data Management (PDM) desktop and web based systems maintain the organizational technical and managerial data to increase the quality of products by improving the processes of development, business process flows, change management,…
Graphical User Interface (GUI) tasks are vital for automating workflows such as software testing, user interface navigation. For users, the GUI is the most intuitive platform for interacting with a computer. Previous work identified a key…
AI-supported tools can help learners overcome challenges in programming education by providing adaptive assistance. However, existing research often focuses on individual tools rather than deriving broader design recommendations. A key…