Related papers: Inventions on selecting GUI elements
Drag and drop operation is one of the key capabilities of any Graphical User Interface. The user can do quite complex operations simply by visually dragging and dropping objects from one location to another. It saves user from remembering…
Aesthetics or "look and feel" is one of the most important features of any graphical user interface. Better aesthetics makes the interface user-friendlier and more popular. Better aesthetics helps the user to understand the meaning of…
Toolbar is one among the popular elements of a graphical user interface. The other popular elements of graphical user interface are buttons, menu, scrollbar, dialog box etc., all of which provide easy access to various functions of a GUI…
Color is an important aspect of any graphical user interface (GUI). Color is used to make a GUI attractive and meaningful. But there are difficulties in using colors too. Improper use of color can result in adverse effects. Wrong colors at…
Operating a GUI through eyeball is a complex mechanism and not used as often as mouse or trackball. But there are situations where eye-mouse devices can play a tremendous role especially where the hands of the user are not available or busy…
Detecting Graphical User Interface (GUI) elements in GUI images is a domain-specific object detection task. It supports many software engineering tasks, such as GUI animation and testing, GUI search and code generation. Existing studies for…
Although a GUI largely replaces textual descriptions by graphical icons, the textual items are not completely removed. The textual items are inevitably used in window titles, message boxes, help items, menu items and popup items. Textual…
The conventional GUI is more mechanical and does not recognize or communicate emotions. The modern GUIs are trying to infer the likely emotional state and personality of the user and communicate through a corresponding emotional state.…
Scrolling mechanism is used to display and edit large documents in a limited screen space or window. The scrolling mechanism may include a vertical scroll bar or a horizontal scroll bar or both to move the contents of the documents up and…
A command button may contain a textual label or a graphic image or both. It may be static or animated. There can be many different features to make a command button attractive and effective. As command button is a typical GUI element, most…
The availability of interaction devices has raised interest in techniques to support the user interface (UI). A UI specification describes the functions that a system provides to its users by capturing the interface details and includes…
Detection of Graphical User Interface (GUI) elements is a crucial task for automatic code generation from images and sketches, GUI testing, and GUI search. Recent studies have leveraged both old-fashioned and modern computer vision (CV)…
Graphical User Interface (GUI) agents, powered by Large Foundation Models, have emerged as a transformative approach to automating human-computer interaction. These agents autonomously interact with digital systems or software applications…
Both toolbar and dropdown menu are used popularly in a graphical user interface with a similar objective of providing easy access to the internal functions. Often the same functions are provided through both menu and toolbar. Both toolbar…
Automation of existing Graphical User Interfaces (GUIs) is important but hard to achieve. Upstream of making the GUI user-accessible or somehow scriptable, even the data-collection to understand the original interface poses significant…
The design and development of robots involve the essential step of selecting and testing robotic interfaces. This interface selection requires careful consideration as the robot's physical embodiment influences and adds to the traditional…
Gestures are a natural form of communication between humans and can also be leveraged for human-robot interaction. This work presents a gesture-based user interface for object selection using pointing and click gestures. An experiment with…
The most important aspect of any Software is the operability for the intended audience. This factor of operability is encompassed in the user interface, which serves as the only window to the features of the system. It is thus essential…
GUI design is an integral part of software development. The process of designing a mobile application typically starts with the ideation and inspiration search from existing designs. However, existing information-retrieval based, and…
AI is powerful, but it can make choices that result in objective errors, contextually inappropriate outputs, and disliked options. We need AI-resilient interfaces that help people be resilient to the AI choices that are not right, or not…