Entropy Aware Message Passing in Graph Neural Networks

Philipp Nazari; Oliver Lemke; Davide Guidobene; Artiom Gesp

Entropy Aware Message Passing in Graph Neural Networks

Machine Learning 2024-03-08 v1

Authors: Philipp Nazari , Oliver Lemke , Davide Guidobene , Artiom Gesp

Abstract

Deep Graph Neural Networks struggle with oversmoothing. This paper introduces a novel, physics-inspired GNN model designed to mitigate this issue. Our approach integrates with existing GNN architectures, introducing an entropy-aware message passing term. This term performs gradient ascent on the entropy during node aggregation, thereby preserving a certain degree of entropy in the embeddings. We conduct a comparative analysis of our model against state-of-the-art GNNs across various common datasets.

Keywords

graph neural network graph generation entropy

Cite

@article{arxiv.2403.04636,
  title  = {Entropy Aware Message Passing in Graph Neural Networks},
  author = {Philipp Nazari and Oliver Lemke and Davide Guidobene and Artiom Gesp},
  journal= {arXiv preprint arXiv:2403.04636},
  year   = {2024}
}

Comments

4 pages, 3 figures

Related papers

View all related →