Graph representations of a target domain often project it to a set of entities (nodes) and their relations (edges). However, such projections often miss important and rich information. For example, in graph representations used in missing value imputation, items - represented as nodes - may contain rich textual information. However, when processing graphs with graph neural networks (GNN), such information is either ignored or summarized into a single vector representation used to initialize the GNN. Towards addressing this, we present CoRGi, a GNN that considers the rich data within nodes in the context of their neighbors. This is achieved by endowing CoRGi's message passing with a personalized attention mechanism over the content of each node. This way, CoRGi assigns user-item-specific attention scores with respect to the words that appear in an item's content. We evaluate CoRGi on two edge-value prediction tasks and show that CoRGi is better at making edge-value predictions over existing methods, especially on sparse regions of the graph.
@article{arxiv.2110.04866,
title = {CoRGi: Content-Rich Graph Neural Networks with Attention},
author = {Jooyeon Kim and Angus Lamb and Simon Woodhead and Simon Peyton Jones and Cheng Zheng and Miltiadis Allamanis},
journal= {arXiv preprint arXiv:2110.04866},
year = {2021}
}