Search papers, labs, and topics across Lattice.
This paper introduces TAG-DLM, a masked diffusion language model that integrates textual reasoning and graph message passing for text-attributed graph learning. By linearizing the local neighborhood of graph instances into token sequences and employing a topology attention mask, the model effectively combines the strengths of language models and graph neural networks. Experimental results demonstrate that TAG-DLM outperforms existing methods, including graph neural networks and LLM-based approaches, achieving up to a 3.9-point improvement on TAG benchmarks across multiple tasks.
A unified model that seamlessly integrates text and graph learning outperforms traditional methods by up to 3.9 points on benchmark tasks.
Text-attributed graphs (TAGs), where each node carries a natural language description, require models to jointly reason over text and graph topology. Existing approaches often handle the two modalities separately: graph neural networks operate on shallow text features, while hybrids of LLMs and graphs use the language model mainly as a text encoder and delegate structure learning to a separate graph module. We propose method that unifies textual reasoning and graph message passing within a masked diffusion language model, a language model with bidirectional attention and generative decoding. For each graph instance, method linearises a sampled local neighbourhood into a token sequence and injects graph structure through a topology attention mask, which realises message passing over the graph. Because the diffusion language model can both interpret and generate text, the method adapts to different tasks simply by changing the prompt, supporting node classification, link prediction, and cross-dataset transfer with no target-specific fine-tuning. Experiments show that method outperforms graph neural networks, graph transformers, and LLM-based baselines on all three TAG benchmarks across two tasks, improving over the strongest baseline by up to 3.9 points.
