相关论文: Edukoi: developing an interactive sonification too…
Some 400 years after Galileo, modern telescopes have enabled humanity to "see" what the natural eye cannot. Astronomical images today contain information about incredibly large objects located across vast distances and reveal information…
Grounding objects in images using visual cues is a well-established approach in computer vision, yet the potential of audio as a modality for object recognition and grounding remains underexplored. We introduce YOSS, "You Only Speak Once to…
Jupyter notebooks are increasingly being adopted by teachers to deliver interactive practical sessions to their students. Notebooks come with many attractive features, such as the ability to combine textual explanations, multimedia content,…
Sonification, or encoding information in meaningful audio signatures, has several advantages in augmenting or replacing traditional visualization methods for human-in-the-loop decision-making. Standard sonification methods reported in the…
Visual to auditory sensory substitution devices convert visual information into sound and can provide valuable assistance for blind people. Recent iterations of these devices rely on depth sensors. Rules for converting depth into sound…
Photo applications offer tools for annotation via text and stickers. Ideophones, mimetic and onomatopoeic words, which are common in graphic novels, have yet to be explored for photo annotation use. We present a method for automatic…
Humans are able to localize objects in the environment using both visual and auditory cues, integrating information from multiple modalities into a common reference frame. We introduce a system that can leverage unlabeled audio-visual data…
The research communities studying visualization and sonification for data display and analysis share exceptionally similar goals, essentially making data of any kind interpretable to humans. One community does so by using visual…
Many species have evolved advanced non-visual perception while artificial systems fall behind. Radar and ultrasound complement camera-based vision but they are often too costly and complex to set up for very limited information gain. In…
One of the major priorities of international radio astronomy is to study the early universe through the detection of the 21 cm HI line from the epoch of reionisation (EoR). Due to the weak nature of the 21 cm signal, an important part in…
We present an interactive IDL program for viewing and analyzing astronomical spectra in the context of modern imaging surveys. SpecPro's interactive design lets the user simultaneously view spectroscopic, photometric, and imaging data,…
In this study we describe a methodology to realize visual images cognition in the broader sense, by a cross-modal stimulation through the auditory channel. An original algorithm of conversion from bi-dimensional images to sounds has been…
Direct exoplanet spectroscopy aims to measure the spectrum of an exoplanet while simultaneously minimizing the light collected from its host star. Isolating the planet light from the starlight improves the signal-to-noise ratio (S/N) per…
Most research data collections created or used by astronomers are intrinsically multi-dimensional. In contrast, all visual representations of data presented within research papers are exclusively 2-dimensional. We present a resolution of…
In the art of video editing, sound helps add character to an object and immerse the viewer within a space. Through formative interviews with professional editors (N=10), we found that the task of adding sounds to video can be challenging.…
The sensory perceptions of vision and sound may be considered as complementary doorways towards interpreting and understanding physical phenomena. We provide a few selected samples where scientific data of systems usually not directly…
Modern sciences and astrophysics in particular study objects and phenomena not visible in physical terms, that is they cannot be investigated with the eyes or analogous optical systems. Nevertheless, they make intensive use of visual…
Movement-sound interactive systems are at the interface of different artistic and educational practices. Within this multiplicity of uses, we examine common denominators in terms of learning, appropriation and relationship to technological…
One of key goals of contemporary physics (and, realistically, STEM) education is to develop students' science literacy and critical thinking skills. In this paper, we present the construction and use of several versions of a simple…
Musicians mostly have to rely on their ears when they want to analyze what they play, for example to detect errors. Since hearing is sequential, it is not possible to quickly grasp an overview over one or multiple recordings of a whole…