Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
Chimaera integrates a Mixture-of-Experts architecture into text-attributed graph foundation models by routing across heterogeneous model backends, including graph prompt tuners and linear GNNs extended to link- and graph-level tasks. This multi-expert design tackles graph foundation modeling's core bottleneck: severe distribution shifts and incompatible embedding spaces across varied graph datasets and tasks. Evaluated on six benchmark datasets, the approach achieves robust few-shot cross-task and cross-dataset transfer, revealing that ensembling simple linear GNNs with multi-scale language model embeddings outperforms monolithic architectures.
Graph foundation models do not require monolithic backends: routing across lightweight linear GNNs conditioned on embeddings from both small and large language models unlocks robust few-shot transfer across node, link, and graph classification tasks.
Designing foundation models for graphs is challenging due to the irregular structure of graphs and the different sizes and characteristics of embeddings. Chimaera integrates mixture-of-experts with graph foundation models (GFM). It integrates different GFM architectures, such as graph prompts and linear GNN models. Large language models are used to generate embeddings, and experts can be trained and combined following different strategies, GFMs, embeddings, etc. Furthermore, Chimaera extends existing linear GNNs to support link-level and graph-level tasks in addition to node-level tasks. Empirical analyses are performed on same-task and cross-task experiments with node, link, and graph classification tasks using six benchmark text-attributed graph datasets. The experiments demonstrate the effectiveness of Chimaera and its capabilities for transfer across tasks and datasets. Further insights include the need to use both large and small language models to generate embeddings for the experts, a strong cross-task transferability of simple but effective linear GNNs, and using few samples only to provide strong results.