Search papers, labs, and topics across Lattice.
This paper introduces ARASH, a novel method for optimizing Tabular Foundation Models (TFMs) by employing adaptive retrieval and shot selection based on local neighborhood analysis of the training set. By leveraging in-context learning through few-shot prompting, ARASH significantly reduces the computational burden associated with TFMs, achieving a remarkable 1261.5脳 reduction in prompt length and a 2.56脳 decrease in memory usage without sacrificing accuracy. These findings highlight a practical approach to enhancing the efficiency of tabular prediction tasks in resource-constrained environments.
ARASH slashes the resource demands of Tabular Foundation Models while maintaining accuracy, making efficient tabular prediction accessible even in low-compute settings.
Tabular prediction is a critical task across numerous applications. The recent success of large language models has sparked various approaches for adapting them to the tabular domain. A prevalent strategy involves training or fine-tuning specialized Tabular Foundation Models (TFMs) such as TabPFN. However, TFMs require substantial computational resources, and frequent model retraining is often impractical. In-context learning (ICL), specifically, few-shot prompting, offers a resource-efficient alternative to enhance performance. Yet, identifying the most relevant rows to serve as shots remains a challenge for tabular data. This paper introduces ARASH (Adaptive, query-specific Retrieval And Shot selection), a method that improves TFM efficiency by selecting optimal shots based on local neighborhood analysis within the training set. Our results demonstrate that ARASH reduces the prompt length and memory usage of TabPFN by 1261.5$\times$ and 2.56$\times$, respectively, while providing comparable accuracy.
