Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper explores the challenges of industrial post-training in large language models (LLMs), emphasizing the need for targeted improvements within fixed compute and mixture budgets. It identifies three key challenges: zero-sum mixture design, yield as a critical metric, and the complexities of end-to-end integration under uncertainty. A case study demonstrates that optimized interventions can significantly enhance the effectiveness of teacher distillation, resulting in notable improvements in code generation performance metrics across multiple benchmarks.
Targeted post-training interventions can boost code generation performance by over 6 points on key benchmarks, challenging the notion that larger models are always better.
Industrial post-training is a brownfield regime. Teams inherit a deployed checkpoint and must land targeted improvements under fixed compute and mixture budgets without regressing the rest. The maintained artifact is increasingly dataware: behavior governed by a curated post-training mixture, updated via bounded mixture patches rather than clean-slate retraining. From an industrial code-generation improvement effort, we offer a maintainer's perspective on why this work is hard in practice, distilling three recurring challenges, zero-sum mixture design, yield as the binding metric, and end-to-end integration under uncertainty, and arguing that progress depends less on one-off recipes than on an engineering discipline for programming dataware. In our case study, interventions that raised the conversion of teacher distillation into usable training data increased accepted supervision by 2.84 times while using the same solution teacher and four solution attempts per candidate problem. In our primary evaluation, the yield-engineered patch improved CodeForces pass@1 by +2.59 points (+3.11 pass@3) and held-out LiveCodeBench v6 pass@1 by +6.11 (+8.05 pass@3), all statistically significant across 16 stochastic evaluations of each benchmark from one fixed checkpoint per condition, with internal AIME and MATH regression suites within tolerance.