Related papers: From Language To Vision: A Case Study of Text Anim…
We give an overview of our philosophy of pictures in mathematics. We emphasize a bi-directional process between picture language and mathematical concepts: abstraction and simulation. This motivates a program to understand different…
Visualizations that adopt a first-person point of view allow observation and, in the case of interactive simulations, experimentation with relativistic scenes. This paper gives examples of three types of first-person visualizations:…
Representing the semantics of words is a long-standing problem for the natural language processing community. Most methods compute word semantics given their textual context in large corpora. More recently, researchers attempted to…
We demonstrate text as a strong cross-modal interface. Rather than relying on deep embeddings to connect image and language as the interface representation, our approach represents an image as text, from which we enjoy the interpretability…
The ability to learn optimal control policies in systems where action space is defined by sentences in natural language would allow many interesting real-world applications such as automatic optimisation of dialogue systems. Text-based…
Our understanding of the visual world goes beyond naming objects, encompassing our ability to parse objects into meaningful parts, attributes, and relations. In this work, we leverage natural language descriptions for a diverse set of 2K…
Recent advances in text-to-image synthesis make it possible to visualize machine imaginations for a given context. On the other hand, when generating text, human writers are gifted at creative visualization, which enhances their writings by…
Many application areas ranging from serious games for health to learning by demonstration in robotics, could benefit from large body movement datasets extracted from textual instructions accompanied by images. The interpretation of…
Grounding language in vision is an active field of research seeking to construct cognitively plausible word and sentence representations by incorporating perceptual knowledge from vision into text-based representations. Despite many…
Inferring universal laws of the environment is an important ability of human intelligence as well as a symbol of general AI. In this paper, we take a step toward this goal such that we introduce a new challenging problem of inferring…
We define "visual story-writing" as using visual representations of story elements to support writing and revising narrative texts. To demonstrate this approach, we developed a text editor that automatically visualizes a graph of entity…
Human reasoning can be understood as a cooperation between the intuitive, associative "System-1" and the deliberative, logical "System-2". For existing System-1-like methods in visual activity understanding, it is crucial to integrate…
Visual Question Answering (VQA) methods have made incredible progress, but suffer from a failure to generalize. This is visible in the fact that they are vulnerable to learning coincidental correlations in the data rather than deeper…
Animacy - whether an entity is alive and sentient - is fundamental to cognitive processing, impacting areas such as memory, vision, and language. However, animacy is not always expressed directly in language: in English it often manifests…
The development of Information and Communication Technologies suggests some spectacular changes in the methods used for teaching scientific subjects. Nowadays, the development of software and hardware makes it possible to simulate processes…
Data visualizations are widely used to communicate messages about urgent topics such as climate change and public health. However, we still know little about how these visualizations are produced and interpreted in popular science contexts.…
Word clouds and text visualization is one of the recent most popular and widely used types of visualizations. Despite the attractiveness and simplicity of producing word clouds, they do not provide a thorough visualization for the…
Speech is one of the interaction modalities that we increasingly come across in natural user interfaces. However, its use in collaborative scenarios has not yet been thoroughly investigated. In this reflection statement, we discuss the…
Narrative visualization aims to communicate scientific results to a general audience and garners significant attention in various applications. Merging exploratory and explanatory visualization could effectively support a non-expert…
This paper describes an immersive and interactive visualization environment that uses light as a metaphor for psychological phenomena. Creative life force is portrayed as ambient light, and peoples' psyches are represented by spheres that…