Embodied Question Answering

Abhishek Das; Samyak Datta; Georgia Gkioxari; Stefan Lee; Devi Parikh; Dhruv Batra

Embodied Question Answering

Computer Vision and Pattern Recognition 2017-12-04 v2 Artificial Intelligence Computation and Language Machine Learning

Authors: Abhishek Das , Samyak Datta , Georgia Gkioxari , Stefan Lee , Devi Parikh , Dhruv Batra

Abstract

We present a new AI task -- Embodied Question Answering (EmbodiedQA) -- where an agent is spawned at a random location in a 3D environment and asked a question ("What color is the car?"). In order to answer, the agent must first intelligently navigate to explore the environment, gather information through first-person (egocentric) vision, and then answer the question ("orange"). This challenging task requires a range of AI skills -- active perception, language understanding, goal-driven navigation, commonsense reasoning, and grounding of language into actions. In this work, we develop the environments, end-to-end-trained reinforcement learning agents, and evaluation protocols for EmbodiedQA.

Keywords

embodied agent

Cite

@article{arxiv.1711.11543,
  title  = {Embodied Question Answering},
  author = {Abhishek Das and Samyak Datta and Georgia Gkioxari and Stefan Lee and Devi Parikh and Dhruv Batra},
  journal= {arXiv preprint arXiv:1711.11543},
  year   = {2017}
}

Comments

20 pages, 13 figures, Webpage: https://embodiedqa.org/

Embodied Question Answering

Abstract

Keywords

Cite

Comments

Related papers