Entropy Aware Message Passing in Graph Neural Networks

Philipp Nazari, Oliver Lemke, Davide Guidobene, Artiom Gesp
3/7/2024
cs.LG

No official code found. Best implementation: oliver-lemke/entropy_aware_message_passing (5 stars, MIT, last commit 2024-03-10).

Abstract

Deep Graph Neural Networks struggle with oversmoothing. This paper introduces a novel, physics-inspired GNN model designed to mitigate this issue. Our approach integrates with existing GNN architectures, introducing an entropy-aware message passing term. This term performs gradient ascent on the entropy during node aggregation, thereby preserving a certain degree of entropy in the embeddings. We conduct a comparative analysis of our model against state-of-the-art GNNs across various common datasets.

View on arXivView PDF

Code Implementations(1)

oliver-lemke/entropy_aware_message_passingCommunity100%
MIT

Repository for the 2023 Deep Learning course project

51PythonOct 31, 20232 years agoMIT

Similar Papers

Channel-Attentive Graph Neural Networks

84% match1 repo42025

Ordered GNN: Ordering Message Passing to Deal with Heterophily and Over-smoothing

84% match2 repos482023

Heterophily-informed Message Passing

84% match1 repo2025

GNN-VPA: A Variance-Preserving Aggregation Strategy for Graph Neural Networks

84% match1 repo102024

Tackling Oversmoothing in GNN via Graph Sparsification: A Truss-based Approach

84% match2024

Transfer Entropy in Graph Convolutional Neural Networks

84% match1 repo32024

Cite this paper

@article{nazari2024entropy,
  title  = {Entropy Aware Message Passing in Graph Neural Networks},
  author = {Philipp Nazari and Oliver Lemke and Davide Guidobene and Artiom Gesp},
  year   = {2024},
  eprint = {2403.04636},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2403.04636v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2403.04636.svg)](https://codeofpaper.com/paper/2403.04636)

Discussion