Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces the Energy-based Representation Alignment (ERAlign) framework, which aligns representations from Graph Neural Networks (GNNs) and Large Language Models (LLMs) on Text-attributed Graphs (TAGs) by projecting them into a shared latent space. The approach addresses the limitations of previous methods that relied on coarse-grained heuristics, leading to representation drift and poor generalization, by employing an Energy-based Model (EBM) objective to optimize layer-wise alignment through a distance metric. Empirical results across eight TAG datasets show that ERAlign achieves state-of-the-art performance, demonstrating its effectiveness in both supervised and cross-task transfer scenarios.
Achieving state-of-the-art alignment between GNNs and LLMs, ERAlign transforms how we leverage textual and structural data in graph-based learning.
Text-attributed Graphs (TAGs) incorporate textual node attributes with graph structures to describe rich relational semantics. Recent efforts to integrate Graph Neural Networks (GNNs) and Large Language Models (LLMs) have shown promise for learning on TAGs, yet achieving well-aligned representations remains challenging. Prior studies largely rely on heuristics that perform coarse-grained matching. They lack sufficient constraints and ignore distributional alignment, leading to representation drift and limited generalization. Building on Energy-based Models (EBMs), we propose an Energy-based Representation Alignment (ERAlign) framework that projects GNN-encoded graph structure and LLM-derived text embeddings in a shared latent space to achieve distribution consistency. Concretely, layer-wise alignment is quantified by a distance metric and optimized via an EBM objective. By decreasing energy values, our framework yields well-aligned representations for downstream tasks. During training, we introduce Energy Discrepancy (ED) to avoid high sampling costs associated with intractable normalization. ED also carries theoretical guarantees of higher training efficiency and reduced energy landscape distortion. Empirical evaluations on eight TAG datasets demonstrate that ERAlign obtains state-of-the-art performance across varying levels of supervision and cross-task transfer scenarios.