Related papers: The Open Vault Challenge -- Learning how to build …

GUI agents drive applications through their visual interfaces instead of programmatic APIs, interacting with arbitrary software via taps, swipes, and keystrokes, reaching a long tail of applications that CLI-based agents cannot. Yet…

This paper describes the ongoing development of a conversational interaction concept that allows visually impaired users to easily create and edit text documents on mobile devices using mainly voice input. In order to verify the concept, a…

Capability evaluations are required to understand and regulate AI systems that may be deployed or further developed. Therefore, it is important that evaluations provide an accurate estimation of an AI system's capabilities. However, in…

The stack-based access control mechanism plays a fundamental role in the security architecture of Java and Microsoft CLR (common language runtime). It is enforced at runtime by inspecting methods in the current call stack for granted…

Simulation models often have parameters as input and return outputs to understand the behavior of complex systems. Calibration is the process of estimating the values of the parameters in a simulation model in light of observed data from…

Experiential AI is an emerging research field that addresses the challenge of making AI tangible and explicit, both to fuel cultural experiences for audiences, and to make AI systems more accessible to human understanding. The central theme…

An ideal software system in computer graphics should be a combination of innovative ideas, solid software engineering and rapid development. However, in reality these requirements are seldom met simultaneously. In this paper, we present…

Natural language is perhaps the most flexible and intuitive way for humans to communicate tasks to a robot. Prior work in imitation learning typically requires each task be specified with a task id or goal image -- something that is often…

This paper introduces EvoCraft, a framework for Minecraft designed to study open-ended algorithms. We introduce an API that provides an open-source Python interface for communicating with Minecraft to place and track blocks. In contrast to…

Robotic assembly is a longstanding challenge, requiring contact-rich interaction and high precision and accuracy. Many applications also require adaptivity to diverse parts, poses, and environments, as well as low cycle times. In other…

Many existing AI music generation tools rely on text prompts, complex interfaces, or instrument-like controls, which may require musical or technical knowledge that non-musicians do not possess. This paper introduces DeformTune, a prototype…

The development of software components for autonomous driving functions should always include an extensive and rigorous evaluation. Since real-world testing is expensive and safety-critical -- especially when facing dynamic racing scenarios…

The cognitive load can be used to assess if someone is struggling while performing a task. It can be used in many different situations such as in driving, piloting, studying, playing, working, etc. This information can help to design better…

Human intelligence emerged through the process of natural selection and evolution on Earth. We investigate what it would take to re-create this process in silico. While past work has often focused on low-level processes (such as simulating…

Although many research vehicle platforms for autonomous driving have been built in the past, hardware design, source code and lessons learned have not been made available for the next generation of demonstrators. This raises the efforts for…

We introduce DrawTalking, an approach to building and controlling interactive worlds by sketching and speaking while telling stories. It emphasizes user control and flexibility, and gives programming-like capability without requiring code.…

Robotics simulation has been an integral part of research and development in the robotics area. The simulation eliminates the possibility of harm to sensors, motors, and the physical structure of a real robot by enabling robotics…

We present Universal Manipulation Interface (UMI) -- a data collection and policy learning framework that allows direct skill transfer from in-the-wild human demonstrations to deployable robot policies. UMI employs hand-held grippers…

Heterogeneous parallel systems are widely spread nowadays. Despite their availability, their usage and adoption are still limited, and even more rarely they are used to full power. Indeed, compelling new technologies are constantly…

One of the open questions in the field of interaction design is "what inputs or interaction techniques should be used with augmented reality devices?" The transition from a touchpad and a keyboard to a multi-touch device was relatively…