Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study investigates the limitations of in-context learning (ICL) in large language models (LLMs) when generating structured data, particularly under distribution mismatch, using high-cardinality tabular data as a benchmark. The authors identify a critical failure mode termed "categorical prior lock-in," where ICL fails to effectively update the model's prior distributions, leading to poor performance on rare classes. While parameter-efficient fine-tuning methods like LoRA can mitigate these issues, they introduce risks of memorization and instability in structured outputs, revealing a trade-off between adaptability and privacy.
Categorical prior lock-in reveals that in-context learning can fail dramatically in structured data scenarios, leaving rare classes unrepresented.
Large language models (LLMs) are increasingly used as conditional generators for structured data, relying on in-context learning (ICL) to adapt to new distributions without parameter updates. We investigate the limits of ICL for structured generation under distribution mismatch, using high-cardinality tabular data as a controlled test case, and identify a structural failure mode we term \textit{categorical prior lock-in}: the inability of ICL to update the model's prior over token distributions inherited from pre-training. Across two 7B-parameter open-weight models, ICL improves numerical fidelity with additional examples but exhibits a sharp ceiling on categorical distributions, failing to reproduce rare classes entirely. Parameter-efficient fine-tuning (LoRA) overcomes these limitations but introduces measurable memorization risk and, in some cases, destabilizes structured output generation, highlighting a fundamental trade-off between adaptability and privacy.