Search papers, labs, and topics across Lattice.
This paper introduces Localized TabICLv2, a method that enhances the efficiency of tabular in-context learning (ICL) by utilizing k-nearest neighbors (k-NN) to reduce inference costs while maintaining accuracy. By retrieving only the closest training examples based on similarity in the model's row-representation space, the approach circumvents the limitations of full-context ICL, which struggles with large datasets due to escalating attention costs. The results demonstrate that the fine-tuned localized model achieves 98.64% of the accuracy of the full model while significantly accelerating batch inference by 2.18 times and single-query serving by approximately 249 times.
Localized TabICLv2 retains nearly all the accuracy of its predecessor while slashing inference time, making it a game-changer for large-scale tabular data tasks.
Foundational models for tabular data have made significant progress in recent years, with TabICLv2 reporting state-of-the-art performance on several tabular classification tasks. However, full-context tabular ICL still suffers from attention cost that grows with the training-context size, which limits its ability to handle large datasets efficiently. Localized TabICLv2 introduces a method that reduces the inference cost of TabICLv2 by retrieving only the k nearest training neighbours for each test point, measured by similarity in the model's Stage 2 row-representation space, rather than using the full training context. This requires no architectural changes, and we show that accuracy retention can be improved through additional Stage 2 and Stage 3 fine-tuning. On TabArena classification tasks, the fine-tuned localized model retains 98.64% of Full TabICLv2 accuracy and it achieves a median 2.18$\times$ speedup in batch inference, and reaches approximately 249$\times$ median speedup in the single-query serving setting.
