Related papers: Integration of Flexible Web Based GUI in I-SOAS
This paper proposes a shift from compliance-centered web accessibility to a care-driven model that prioritizes user autonomy, using neurodivergent users as a catalyst case for broader personalization needs. While accessibility standards…
Modern physics experiments often utilize FPGA-based systems for real-time data acquisition. Integrated analog electronics demand for complex calibration routines. Furthermore, versatile configuration and control of the whole system is a key…
Advances in large language models (LLMs) and real-time speech recognition now make it possible to issue any graphical user interface (GUI) action through natural language and receive the corresponding system response directly through the…
Recent progress in GUI agents has substantially improved visual grounding, yet robust planning remains challenging, particularly when the environment deviates from a canonical initial state. In real applications, users often invoke…
Traditional static user interfaces (UI) have given way to dynamic systems that can intelligently adapt to and respond to users' changing needs. Temporal interaction is an emerging field in human-computer interaction (HCI), which refers to…
Socially interactive agents (SIAs) are no longer mere visions for future user interfaces, as 20 years of research and technology development has enabled the use of virtual and physical agents in day-to-day interfaces and environments. This…
We report on our experiences in an e-government project for supporting the automatic generation of E-forms for services provided by local governments. The approach requires the integration of both the model-based user interface design…
Recent advances in vision-language models (VLMs) and reinforcement learning (RL) have driven progress in GUI automation. However, most existing methods rely on static, one-shot visual inputs and passive perception, lacking the ability to…
As generative artificial intelligence advances, Large Language Models (LLMs) are being explored for automated graphical user interface (GUI) design. This study investigates the usability and adaptability of LLM-generated interfaces by…
Graphical User Interface (GUI) grounding - the task of mapping natural language instructions to screen coordinates - is essential for autonomous agents and accessibility technologies. Existing systems rely on monolithic models or one-shot…
Recent advances in computing systems have led to a new digital era in which every area of life is nearly interrelated with information technology. However, with the trend towards large-scale IT systems, a new challenge has emerged. The…
This paper presents HyperGraphOS, an innovative Operating System designed for the scientific and engineering domains. It combines model based engineering, graph modeling, data containers, and computational tools, offering users a dynamic…
Developing web applications requires dealing with their distributed nature and the natural asynchronicity of user input and network communication. For facilitating this, different researchers have explored the combination of a multi-tier…
Over the last decades, the amount of data of all kinds available electronically has increased dramatically. Data are accessible through a range of interfaces including Web browsers, database query languages, application-specific interfaces,…
The performance and generalization of foundation models for interactive systems critically depend on the availability of large-scale, realistic training data. While recent advances in large language models (LLMs) have improved GUI…
The analysis of the current integration attempts of some modes and use cases of user-machine interaction is presented. The new concept of the user-driven intelligent interface is proposed on the basis of multimodal augmented reality and…
The automation of user interface development has the potential to accelerate software delivery by mitigating intensive manual implementation. Despite the advancements in Large Multimodal Models for design-to-code translation, existing…
In immersive augmented reality (IAR), users can wear a head-mounted display to see computer-generated images superimposed to their view of the world. IAR was shown to be beneficial across several domains, e.g., automotive, medicine, gaming…
Building Graphical User Interface (GUI) agents is a promising research direction, which simulates human interaction with computers or mobile phones to perform diverse GUI tasks. However, a major challenge in developing generalized GUI…
The most important aspect of any Software is the operability for the intended audience. This factor of operability is encompassed in the user interface, which serves as the only window to the features of the system. It is thus essential…