Related papers: The Command Line GUIde: Graphical Interfaces from …
Recent advancements in eye tracking technology are driving the adoption of gaze-assisted interaction as a rich and accessible human-computer interaction paradigm. Gaze-assisted interaction serves as a contextual, non-invasive, and explicit…
Humans communicate with graphical sketches apart from symbolic languages. Primarily focusing on the latter, recent studies of emergent communication overlook the sketches; they do not account for the evolution process through which symbolic…
For instruments with many occasional users, it is important to have easy to use software. To support the frequent users it is important to be flexible. Using a scripting language to design a GUI and exposing it to the user allows us to do…
Humans possess a large amount of, and almost limitless, visual memory, that assists them to remember pictures far better than words. This phenomenon has recently motivated the computer security researchers' in academia and industry to…
Adaptive interfaces can help users perform sequential decision-making tasks like robotic teleoperation given noisy, high-dimensional command signals (e.g., from a brain-computer interface). Recent advances in human-in-the-loop machine…
Chat-based prompts respond with verbose linear-sequential texts, making it difficult to explore and refine ambiguous intents, back up and reinterpret, or shift directions in creative AI-assisted design work. AI-Instruments instead embody…
In this paper, we discuss the generation of symbols (and alphabets) based on specific user requirements (medium, priorities, type of information that needs to be conveyed). A framework for the generation of alphabets is proposed, and its…
We present a workflow and associated toolkit to automate the creation of graphical user interfaces (GUI) for executables run from command line interfaces (CLI). The workflow consists of three phases, namely (Step 1) the plugin design, (Step…
While Graphical User Interface (GUI) agents have shown promising performance in automated device interaction, they primarily depend on static parametric knowledge from pre-training or instruction tuning. This reliance fundamentally limits…
This paper presents the design and implementation of a Human Interface for a housekeeper robot. It bases on the idea of making the robot understand the human needs without making the human go through the details of robots work, for example,…
As the boundaries of human computer interaction expand, Generative AI emerges as a key driver in reshaping user interfaces, introducing new possibilities for personalized, multimodal and cross-platform interactions. This integration…
Automated understanding of user interfaces (UIs) from their pixels can improve accessibility, enable task automation, and facilitate interface design without relying on developers to comprehensively provide metadata. A first step is to…
Over the past half century, there have been several false dawns during which the "arrival" of world-changing artificial intelligence (AI) has been heralded. Tempting fate, the authors believe the age of AI has, indeed, finally arrived.…
Command line learning and Bash usage are fundamental skills in systems administration, software development, and data science environments. However, their teaching has been neglected in many curricula, despite its relevance in the…
Many terminals are used in safety-critical operations in which humans, through terminal user interfaces, become a part of the system control loop (e.g., medical and industrial systems). These terminals are typically embedded, single-purpose…
This paper presents 3Description, an experimental human-AI collaborative approach for intuitive 3D modeling. 3Description aims to address accessibility and usability challenges in traditional 3D modeling by enabling non-professional…
With the development of artificial intelligence, writing assistants (WAs) are changing the way people interact with text, creating lengthy outputs that can be overwhelming for users. The programming field has long addressed this issue, and…
Visual analysis of relational data is essential for many real-world analytics tasks, with layout quality being key to interpretability. However, existing layout algorithms often require users to navigate complex parameters to express their…
GUI agents are rapidly becoming a new interaction to software, allowing people to navigate web, desktop and mobile rather than execute them click by click. Yet ``agent'' is described with radically different degrees of autonomy, obscuring…
The field of artificial intelligence has seen explosive growth and exponential success. The last phase of development showcased deep learnings ability to solve a variety of difficult problems across a multitude of domains. Many of these…