Search papers, labs, and topics across Lattice.
This paper introduces TabMF, a framework that repurposes pretrained tabular foundation models (TFMs) as encoders for supervised datasets, enabling higher-level ML tasks. By equipping TFMs with an architecture-aware readout and adapting them via contrastive post-training on synthetic data, TabMF generates powerful dataset representations. The resulting encoders achieve state-of-the-art performance on tasks like dataset retrieval and distribution-shift assessment, demonstrating the inherent dataset-level structure captured by predictive TFM pretraining.
Pretrained tabular foundation models, when adapted with a simple readout and contrastive learning, unlock state-of-the-art dataset representations for retrieval and shift detection.
Representations of supervised datasets support higher-level machine-learning tasks such as dataset retrieval and distribution-shift monitoring. However, existing approaches typically rely either on hand-crafted meta-features or on dataset encoders trained specifically for representation learning. We introduce TabMF, a framework that repurposes pretrained tabular foundation models (TFMs) as encoders of supervised datasets. TabMF equips a predictive TFM with an architecture-aware dataset-level readout and adapts the model end-to-end using contrastive post-training on stochastic views of synthetic supervised datasets. We instantiate TabMF on TabICLv2 and TabPFN-2.5 and evaluate the resulting representations across six tasks on unseen real datasets. Simple frozen TFM readouts yield competitive representations, indicating that predictive pretraining already induces substantial dataset-level structure. The two TabMF encoders obtain the strongest aggregate performance among the evaluated learned and classical dataset representations, with particularly strong results in dataset retrieval, compact benchmark construction, and distribution-shift severity assessment.
