Search papers, labs, and topics across Lattice.
This paper addresses the challenge of self-hosting large language models (LLMs) within enterprises constrained by data residency requirements, by consolidating traffic from over 200 applications onto a single model. The authors identify and close quality gaps through production error analysis across instruction following, function-calling, and internal task distribution, utilizing a specialized training approach that mitigates reward interference. The resulting model significantly outperforms a baseline with 7 times more parameters, efficiently managing 50% of platform traffic while reducing serving costs.
A single self-hosted LLM can absorb 50% of enterprise traffic while outperforming larger models in critical quality metrics.
Data-residency constraints force enterprises to self-host LLMs, but continuous adoption of newer models without decommissioning their predecessors expands the serving fleet, fragmenting a finite GPU pool. We consolidate traffic from over 200 internal applications onto a single model by closing quality gaps identified through production error analysis along three axes: instruction following, function-calling, and internal task distribution. Quality is tracked by offline benchmarks stratified to production traffic and scored by deterministic verifiers or calibrated LLM judges. Rather than optimising all objectives jointly, which introduces cross-domain reward interference, we train a separate GRPO expert per axis and merge them via two-stage SLERP. Each expert's reward exposes a distinct failure mode, namely semantic collapse, over-calling, and verbosity hacking, each requiring a domain-specific fix. In non-reasoning mode the recipe surpasses a ${\sim}7\times$ larger by total parameters baseline on the in-house Arena with 69.6 to 65.8, instruction following with 0.85 to 0.83, and function-calling with 0.79 to 0.77, while lifting general dialogue benchmarks. The model absorbs 50% of platform traffic, 116M requests per month, at a fraction of the serving cost.
