Functional Object-Oriented Network for Manipulation Learning
Abstract
This paper presents a novel structured knowledge representation called the functional object-oriented network (FOON) to model the connectivity of the functional-related objects and their motions in manipulation tasks. The graphical model FOON is learned by observing object state change and human manipulations with the objects. Using a well-trained FOON, robots can decipher a task goal, seek the correct objects at the desired states on which to operate, and generate a sequence of proper manipulation motions. The paper describes FOON's structure and an approach to form a universal FOON with extracted knowledge from online instructional videos. A graph retrieval approach is presented to generate manipulation motion sequences from the FOON to achieve a desired goal, demonstrating the flexibility of FOON in creating a novel and adaptive means of solving a problem using knowledge gathered from multiple sources. The results are demonstrated in a simulated environment to illustrate the motion sequences generated from the FOON to carry out the desired tasks.
Cite
@article{arxiv.1902.01537,
title = {Functional Object-Oriented Network for Manipulation Learning},
author = {David Paulius and Yongqiang Huang and Roger Milton and William D. Buchanan and Jeanine Sam and Yu Sun},
journal= {arXiv preprint arXiv:1902.01537},
year = {2020}
}
Comments
IROS 2016 Submission -- Corrected several errors from the published version (last updated November 28th, 2020)