Related papers: 10 Inventions on scrolling and scrollbars in Graph…
The shape and size of the objects, which we see on the screen, when the application is running, are defined at the design time. By using some sort of adaptive interface, developers give users a chance to resize these objects or on rare…
Graphical user interface (GUI) grounding, the process of mapping human instructions to GUI actions, serves as a fundamental basis to autonomous GUI agents. While existing grounding models achieve promising performance to simulate the mouse…
Large curved displays are ideal for viewing 360 degree content, such as 3D maps, but typically restrict users to a 180 degree viewport, leaving information off-screen. Since users naturally direct their heads toward regions on-screen before…
On modern computers with graphical user interfaces, application windows are managed by a window manager, a core component of the desktop environment. Mainstream operating systems such as Microsoft Windows and Apple's macOS employ window…
The soft keyboards are onscreen representation of physical keyboard having alphanumeric characters and other controls. The user operates the soft keyboard with the mouse, a stylus or other pointing device. The soft keys dont have any…
We are communicating with computers on two different levels. On upper level we have a very flexible system of windows: we can move them, resize, overlap or put side by side. At any moment we decide what would be the best view and reorganize…
The computer industry is developing at a fast pace. With this development almost all of the fields under computers have advanced in the past couple of decades. But the same technology is being used for human computer interaction that was…
Virtual displays provided through head-worn displays (HWDs) offer users large screen space for productivity, but managing this space effectively presents challenges. This paper explores how to enhance window-switching strategies for virtual…
Since computer was invented, people are using many devices to interact with computer. Initially there were keyboard, mouse etc. but with the advancement of technology, new ways are being discovered that are quite usual and natural to the…
A keyboard is the most important input device for a computer. It is used with various types and sizes of computer. But the same standard keyboard will not work efficiently with different types of computers at different environments. There…
People often struggle to interpret data with extremely large or small values, or ranges spanning multiple orders of magnitude. While traditional approaches, such as log scales and multiscale visualizations, can help, we explore in this…
With current development universally in computing, now a days user interaction approaches with mouse, keyboard, touch-pens etc. are not sufficient. Directly using of hands or hand gestures as an input device is a method to attract people…
This paper presents the maneuver of mouse pointer and performs various mouse operations such as left click, right click, double click, drag etc using gestures recognition technique. Recognizing gestures is a complex task which involves many…
We demonstrate EyeDoc, a tool for navigating software documentation with the use of the eyes. When programming, developers often have many windows open such as an IDE, consoles and GUIs for third-party utilities, the application under…
Interactive high-performance computing is doubtlessly beneficial for many computational science and engineering applications whenever simulation results should be visually processed in real time, i.e. during the computation process.…
Understanding user interface (UI) functionality is a useful yet challenging task for both machines and people. In this paper, we investigate a machine learning approach for screen correspondence, which allows reasoning about UIs by mapping…
Graphical User Interface (GUI) agents, powered by Large Foundation Models, have emerged as a transformative approach to automating human-computer interaction. These agents autonomously interact with digital systems or software applications…
Judging the readability of text has many important applications, for instance when performing text simplification or when sourcing reading material for language learners. In this paper, we present a 518 participant study which investigates…
Eye tracking has been a pivotal tool in diverse fields such as vision research, language analysis, and usability assessment. The majority of prior investigations, however, have concentrated on expansive desktop displays employing…
A keyboard is the most important input device for a computer. With the development of technology a basic keyboard does not want to remain confined within the basic functionalities of a keyboard, rather it wants to go beyond. There are…