Graphs are a useful abstraction of image content. Not only can graphs represent details about individual objects in a scene but they can capture the interactions between pairs of objects. We present a method for training a convolutional neural network such that it takes in an input image and produces a full graph definition. This is done end-to-end in a single stage with the use of associative embeddings. The network learns to simultaneously identify all of the elements that make up a graph and piece them together. We benchmark on the Visual Genome dataset, and demonstrate state-of-the-art performance on the challenging task of scene graph generation.
@article{arxiv.1706.07365,
title = {Pixels to Graphs by Associative Embedding},
author = {Alejandro Newell and Jia Deng},
journal= {arXiv preprint arXiv:1706.07365},
year = {2018}
}
Comments
Updated numbers. Code and pretrained models available at https://github.com/umich-vl/px2graph