Search papers, labs, and topics across Lattice.
This study evaluates the effectiveness of TabPFN as a zero-gradient classification head for frozen multimodal encoders, addressing the calibration issues prevalent in traditional lightweight heads like k-nearest neighbors and logistic regression. Through extensive testing across 22,820 episodes and 14 datasets, TabPFN outperformed nine other classification heads, achieving significant reductions in negative log-likelihood (NLL) and expected calibration error (ECE) while maintaining competitive accuracy. The findings indicate that TabPFN is particularly advantageous in scenarios with moderate-to-high shot counts and low-to-moderate feature dimensions, providing a robust solution for calibration-sensitive applications in multimodal classification.
TabPFN slashes negative log-likelihood by up to 62% and drastically improves calibration in multimodal classification without requiring any training.
Few-shot multimodal classification commonly attaches a lightweight head, such as $k$-nearest neighbors, logistic regression, or a linear SVM, to a frozen pretrained encoder. Although computationally efficient, these heads can produce poorly calibrated confidence scores, limiting their reliability in calibration-sensitive applications. We evaluate TabPFN as a plug-and-play, zero-gradient classification head for frozen image, text, and audio encoders. Across 22{,}820 evaluation episodes spanning 14 datasets, 11 encoders, and three modalities, TabPFN achieves the best mean rank among nine classification heads on both negative log-likelihood (NLL) and expected calibration error (ECE). At a representative setting, it reduces NLL by 48--62\% and ECE by 2.1--5.3$\times$ relative to the average of the eight baselines while matching or exceeding their average accuracy. Its accuracy advantage is conditional, concentrating at moderate-to-high shot counts and low-to-moderate feature dimensions ($k \ge 50$, $d \le 32$), and diminishing when labeled data are scarce, feature dimensions are high, or competing methods approach ceiling accuracy. In targeted backbone-adaptation experiments, replacing the trained linear head with TabPFN substantially improves calibration while preserving competitive accuracy. These results provide empirical guidance for using TabPFN as a training-free head in calibration-sensitive multimodal classification. To support transparency and reproducibility, we publicly release the source code, experiment configurations, and evaluation scripts in our GitHub repository: https://github.com/Jingxiang-Zhang/tabpfn-multimodal-embeddings.
