Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces CASE (Context-Aware Semantic Embeddings), a framework that enhances tabular learners by integrating semantic understanding from Large Language Models (LLMs) with statistical learning. By utilizing a contextualization strategy that pre-fills the KV cache of a custom-trained Gemma 3-based Tabular Language Model, CASE dynamically contextualizes row embeddings, effectively resolving semantic ambiguities. Experimental results across benchmarks such as CARTE, TextTab, and TabArena show significant performance improvements for tabular learners, especially in scenarios with limited data.
Contextualizing row embeddings with LLMs boosts tabular learner performance by resolving semantic ambiguities, particularly in low-data settings.
While modern tabular learners excel at capturing statistical patterns, they frequently operate in a semantic vacuum, treating textual features as discrete symbols, ignoring the rich semantics inherent in feature names or cell entries. We propose CASE (Context-Aware Semantic Embeddings), a novel framework that bridges the gap between the semantic understanding of Large Language Models (LLMs) and the statistical capabilities of tabular learners. Unlike existing methods that embed rows in isolation, CASE utilizes a contextualization strategy: we pre-fill the KV cache of a custom-trained Gemma 3-based Tabular Language Model with a representative sample of rows to establish a persistent anchor of the dataset's semantics. This ensures that generated row embeddings are dynamically contextualized, resolving semantic ambiguities and anchoring representations in domain-specific context. Our experiments across several benchmarks (CARTE, TextTab, and TabArena) demonstrate that CASE substantially improves the performance of tabular learners on semantically rich datasets, particularly in low-data regimes.