Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces CHARM, a multimodal graph foundation model designed to facilitate zero-shot transfer across diverse graph domains by employing hierarchical context modeling. By replacing isolated raw nodes with hierarchical graph contexts that encapsulate multimodal semantics and cross-modal relations, CHARM effectively maps domain-specific patterns to shared high-level concepts, minimizing the need for target-domain supervision. Experimental results demonstrate that CHARM consistently outperforms existing methods on zero-shot multimodal graph tasks, highlighting its potential for broader applicability in real-world scenarios.
CHARM achieves zero-shot transfer in multimodal graphs by transforming raw node representations into hierarchical contexts, significantly enhancing cross-domain knowledge sharing.
Graph foundation models (GFMs) have emerged as a promising paradigm for transferring knowledge across graph domains and tasks. Real-world graphs associate nodes with text, images, and other modalities, making multimodal graphs essential for representing complex entities and relations. Moreover, collecting labels and adapting models for every new graph domain is costly and often infeasible, motivating zero-shot transfer. Unfortunately, zero-shot transfer on multimodal graphs remains underexplored. Existing GNN-based graph foundation models typically require downstream adaptation, whereas LLM-based graph methods mainly address unimodal graphs or tasks within a single domain. This setting presents two key challenges. First, models must generalize knowledge from individual modalities while capturing transferable cross-modal relations. Second, without target-domain fine-tuning, node representations remain entangled with domain-specific structures and modality-specific characteristics, obscuring shared concepts in unseen domains. To address these challenges, we propose CHARM, a multimodal graph foundation model with hierarchical context modeling for zero-shot transfer. CHARM replaces isolated raw nodes with hierarchical graph contexts that capture multimodal semantics and cross-modal relations. These contexts map domain-specific node patterns to shared high-level concepts, reducing reliance on target-domain supervision or adaptation. A modality-aware graph context encoder integrates multimodal information with graph structure and converts the resulting representations into graph tokens for a large language model . Experiments show consistent improvements on zero-shot multimodal graph tasks.