Related papers: Body as controller
Mobile devices' user interfaces are still quite similar to traditional interfaces offered by desktop computers, but those can be highly problematic when used in a mobile context. Human gesture recognition in mobile interaction appears as an…
The paper describes results on two components of a research program focused on motion-based communication mediated by the dynamics of a control system. Specifically we are interested in how mobile agents engaged in a shared activity such as…
The quality of a simulator equipped with a haptic interface is given by the dynamical properties of its components: haptic interface, simulator and control system. Some application areas of such kind of simulator like musical synthesis,…
This paper presents the soma design process of creating Body Electric: a novel interface for the capture and use of biofeedback signals and physiological changes generated in the body by breathing, during singing. This NIME design is…
We introduce a real-time, human-in-the-loop gesture control framework that can dynamically adapt audio and music based on human movement by analyzing live video input. By creating a responsive connection between visual and auditory stimuli,…
Position: (1) Partial solutions to machine intelligence can lead to systems which may be useful creating interesting and expressive musical works. (2) An appropriate general goal for this field is augmenting human expression. (3) The study…
Humanoid robots have seen remarkable advances in dexterity, balance, and locomotion, yet their role in expressive domains such as music performance remains largely unexplored. Musical tasks, like drumming, present unique challenges,…
The human-machine interface is of critical importance for master-slave control of the robotic system for surgery, in which current systems offer the control or two robotic arms teleoperated by the surgeon's hands. To relax the need for…
The hand is one of the most complex and important parts of the human body. The dexterity provided by its multiple degrees of freedom enables us to perform many of the tasks of daily living which involve grasping and manipulating objects of…
We examine a class of techniques for 3D object manipulation on mobile devices, in which the device's physical motion is applied to 3D objects displayed on the device itself. This "local coupling" between input and display creates specific…
The hand, a complex effector comprising dozens of degrees of freedom of movement, endows us with the ability to flexibly, precisely, and effortlessly interact with objects. The neural signals associated with dexterous hand movements in…
Sound synthesiser controls typically correspond to technical parameters of signal processing algorithms rather than intuitive sound descriptors that relate to human perception of sound. This makes it difficult to realise sound ideas in a…
Live interactions have the potential to meaningfully engage audiences during musical performances, and modern technologies promise unique ways to facilitate these interactions. This work presents findings from three co-design sessions with…
Artistic creation is often seen as a uniquely human endeavor, yet robots bring distinct advantages to music-making, such as precise tempo control, unpredictable rhythmic complexities, and the ability to coordinate intricate human and robot…
Hand gesture recognition has been granted as one of the emerging fields in research today providing a natural way of communication between man and a machine. Gestures are some forms of body motions which a person expresses when doing a work…
In this work, we propose a gesture based language to allow humans to interact with robots using their body in a natural way. We have created a new gesture detection model using neural networks and a custom dataset of humans performing a set…
As robots enter everyday spaces like offices, the sounds they create affect how they are perceived. We present Music Mode, a novel mapping between a robot's joint motions and sounds, programmed by artists and engineers to make the robot…
The violin is one of the most popular musical instruments. Various parameters of bowing motion, such as pressure, position, and speed, are crucial for producing a beautiful tone. However, mastering them is challenging and requires extensive…
In this paper we look at some of the design issues that affect the success of multimodal displays that combine acoustic and haptic modalities. First, issues affecting successful sonification design are explored and suggestions are made…
The conception and design of new musical interfaces is a multidisciplinary area that tightly relates technology and artistic creation. In this paper, the author first exposes some of the questions he has posed himself during more than a…