STAGE: Simplified Text-Attributed Graph Embeddings Using Pre-trained LLMs

Aaron Zolnai-Lucas, Jack Boylan, Chris Hokamp, Parsa Ghaffari
7/10/2024
cs.CLcs.AI

No official code found. Best implementation: aaronzo/STAGE (9 stars, MIT, last commit 2024-05-24).

Abstract

We present Simplified Text-Attributed Graph Embeddings (STAGE), a straightforward yet effective method for enhancing node features in Graph Neural Network (GNN) models that encode Text-Attributed Graphs (TAGs). Our approach leverages Large-Language Models (LLMs) to generate embeddings for textual attributes. STAGE achieves competitive results on various node classification benchmarks while also maintaining a simplicity in implementation relative to current state-of-the-art (SoTA) techniques. We show that utilizing pre-trained LLMs as embedding generators provides robust features for ensemble GNN training, enabling pipelines that are simpler than current SoTA approaches which require multiple expensive training and prompting stages. We also implement diffusion-pattern GNNs in an effort to make this pipeline scalable to graphs beyond academic benchmarks.

View on arXivView PDF

Code Implementations(1)

aaronzo/STAGECommunity100%
MIT
90Shell, PythonMay 10, 20242 years agoMIT

Similar Papers

LLM as GNN: Graph Vocabulary Learning for Text-Attributed Graph Foundation Models

90% match1 repo252025

GaLoRA: Parameter-Efficient Graph-Aware LLMs for Node Classification

90% match2026

Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs

90% match1 repo3162023

Can GNN be Good Adapter for LLMs?

90% match2 repos262024

Parameter-Efficient Tuning Large Language Models for Graph Representation Learning

89% match2024

Leveraging Large Language Models for Node Generation in Few-Shot Learning on Text-Attributed Graphs

89% match1 repo102023

Cite this paper

@article{zolnailucas2024stage,
  title  = {STAGE: Simplified Text-Attributed Graph Embeddings Using Pre-trained LLMs},
  author = {Aaron Zolnai-Lucas and Jack Boylan and Chris Hokamp and Parsa Ghaffari},
  year   = {2024},
  eprint = {2407.12860},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2407.12860v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2407.12860.svg)](https://codeofpaper.com/paper/2407.12860)

Discussion