Search papers, labs, and topics across Lattice.
This paper addresses the issue of model collapse in iterative instruction tuning, where training on synthetic data can lead to performance degradation due to biased reinforcement of strong skills at the expense of weaker ones. The authors introduce KITE (Knowledge-boundary Instruction Tuning via Exploration), a two-stage framework that utilizes failure-guided data generation and boundary-aware uncertainty curation to mitigate this polarization of competence. Experimental results demonstrate that KITE achieves more stable improvements compared to existing synthetic data approaches across various datasets and open-source LLMs.
Model collapse isn't just about performance loss; it can polarize competence, but KITE offers a solution that ensures consistent improvement in LLMs.
Model collapse is a central challenge in learning from synthetic data: as later-generation large language models (LLMs) are trained on an increasing proportion of model-generated data, performance can degrade due to narrowed coverage and accumulated bias. Existing work mainly studies how to bound this degradation. In iterative model evolution, however, the more meaningful objective is to ensure that each successive model improves over its predecessor, which requires diagnosing collapse at a granularity that is actionable for data curation. We study this problem in synthetic data self-improving for instruction tuning. We show that collapse in this setting is not simply uniform performance degradation, but can appear as a polarization of competence, where synthetic training reinforces already strong skills while further degrading weak ones. Motivated by this observation, we propose KITE (Knowledge-boundary Instruction Tuning via Exploration), a two-stage framework that combines failure-guided data generation with boundary-aware uncertainty curation. Experiments across several datasets and multiple open-source LLMs show that KITE yields more stable improvement than strong synthetic-data baselines.
