Search papers, labs, and topics across Lattice.
This paper introduces TL-ANDI, a novel framework for Context-Constrained Transfer Learning that addresses the limitations of Tabular Foundation Models (TFMs) in handling strict context-size constraints and distribution shifts. By employing a budget-constrained optimal transport problem, TL-ANDI effectively selects a compact source context that optimally balances target covariate coverage and posterior compatibility. The method demonstrates significant improvements in transfer learning performance, mitigating negative transfer effects and enhancing model adaptability across tasks.
TL-ANDI transforms how Tabular Foundation Models handle transfer learning by optimizing context selection to prevent negative transfer.
Tabular Foundation Models (TFMs) have demonstrated strong empirical performance as black-box inference engines through in-context learning. However, their use in transfer learning is limited by two obstacles: strict context-size constraints and sensitivity to distribution shifts between source and target tasks. Directly pooling heterogeneous source data can therefore lead to negative transfer. To address these challenges, we propose Context-Constrained Transfer Learning via ANchoring and DIstillation (TL-ANDI), a posterior-aware distillation framework for TFMs. TL-ANDI constructs a compact source context by solving a budget-constrained optimal transport problem whose cost jointly measures target covariate coverage and posterior compatibility. The selected anchor samples are then equipped with locally distilled labels and combined with a residual calibration step using target data.
Tsinghua AI