Search papers, labs, and topics across Lattice.
This paper introduces GraspLLM, a novel framework that enhances zero-shot generalization on Text-Attributed Graphs (TAGs) by integrating the structural comprehension of graphs with the semantic capabilities of Large Language Models (LLMs). By employing motif-aware contrastive learning and an optimal contextual subgraph extraction method, GraspLLM effectively captures transferable structural patterns across diverse datasets and tasks. Experimental results demonstrate that GraspLLM significantly outperforms existing LLM-based approaches, particularly in zero-shot settings, underscoring its potential for broader applications in real-world scenarios involving TAGs.
GraspLLM achieves unprecedented zero-shot generalization on Text-Attributed Graphs, outperforming existing methods by effectively merging graph structure with LLM semantics.
Research on Text-Attributed Graphs (TAGs) has gained significant attention recently due to its broad applications across various real-world data scenarios, such as citation networks, e-commerce platforms, social media, and web pages. Inspired by the remarkable semantic understanding ability of Large Language Models (LLMs), there have been numerous attempts to integrate LLMs into TAGs. However, existing methods still struggle to generalize across diverse graphs and tasks, and their ability to capture transferable graph structural patterns remains limited. To address this, we introduce the GraspLLM, a framework that combines Graph structural comprehension with semantic understanding prowess of LLMs to enhance the cross-dataset and cross-task generalizability. Specifically, we represent node texts from different graphs in a unified semantic space with a frozen general embedding model, on top of which we perform motif-aware contrastive learning across multiple motif-induced adjacency matrices to extract dataset-agnostic structural information. Then, with our proposed optimal contextual subgraph, we extract the most contextually relevant subgraph for each target node and align these subgraphs to the token space of LLM via an alignment projector. Extensive experiments on TAG benchmark datasets spanning diverse domains reveal that GraspLLM consistently outperforms previous LLM-based methods for TAGs, especially in zero-shot scenarios, highlighting its strong generalizability across different datasets and tasks. Our code is available at https://github.com/Heinz217/GraspLLM.
