Search papers, labs, and topics across Lattice.
This paper introduces DAG-FM, a novel foundation model designed for causal discovery that addresses the challenges posed by heterogeneous causal mechanisms in observational tabular data. By employing a two-stage auto-regressive approach with specialized Transformer-based sub-modules, DAG-FM effectively decomposes the causal discovery process and utilizes a Mixture-of-Leaf-Experts (MoLE) framework to adapt to varying Functional Causal Model assumptions. Extensive experiments reveal that DAG-FM outperforms traditional methods and recent foundation models in both accuracy and scalability on a range of synthetic and real-world datasets.
DAG-FM achieves state-of-the-art causal discovery performance by dynamically adapting to diverse causal mechanisms, outperforming both classical algorithms and recent models.
Causal discovery from observational tabular data remains fundamentally challenging, primarily due to the heterogeneity of underlying causal mechanisms and the high-dimensional combinatorial search space of Directed Acyclic Graphs (DAGs). In this paper, we propose \textbf{DAG-FM}, a novel foundation model architecture that amortizes causal discovery. Unlike direct matrix prediction, DAG-FM decomposes the causal discovery process into two auto-regressive stages using two specialized Transformer-based sub-modules: a leaf-node predictor and a parent-node predictor. To effectively model complex row-column interactions, we adopt a robust tabular interaction block to output feature-wise representations. Crucially, to handle diverse and unknown Functional Causal Model (FCM) assumptions in real-world scenarios, we introduce Mixture-of-Leaf-Experts (MoLE), allowing the model to dynamically route and adapt to identifiable mechanism families. Through an iterative inference algorithm, DAG-FM seamlessly extracts causal orderings and constructs valid DAGs. Extensive experiments demonstrate that DAG-FM achieves state-of-the-art performance on both synthetic benchmarks and complex real-world datasets, significantly outperforming traditional classical algorithms and recent foundation models in both accuracy and scalability.
Tsinghua AI