Search papers, labs, and topics across Lattice.
The authors conduct a controlled failure analysis of correctness-gated multi-teacher distillation across eight experimental arms using a 63.9M-parameter student model to decouple candidate decision correctness from factual rationale grounding. While correctness-weighted distillation nominally improves accuracy (+0.166) and reduces conditional unsafe actions (-0.498) over unfiltered distillation, it induces severe label collapse鈥攑roducing zero recall for the "Refuted" class across all seeds and predicting "NotEnoughInfo" for all claim examples in two seeds. Ultimately, the weighted scheme provides no incremental decision advantage over simple hard filtering, and an empirical audit demonstrates that nearly all student positives still contain unsupported hallucinations.
Gating multi-teacher distillation on candidate correctness induces catastrophic label collapse and zero minority-class recall while failing to outperform simple hard filtering or improve factual grounding.
Candidate decision correctness and rationale grounding are different objectives. We examine correctness-gated multi-teacher distillation in a fixed experiment. Eight arms share 4,330 sources, a 63.9M-parameter student, 12,990 optimization rows, 406 updates, evidence inputs, and a decoder; seven teacher-based arms use one fixed three-response pool. Three seeds are evaluated on 267 held-out examples. Relative to unfiltered distillation, the correctness-weighted arm differed in accuracy by +0.1660 (95% observed-matrix interval [0.0670, 0.2455]), five-label macro-F1 by +0.1323 ([0.0916, 0.1731]), and task-defined conditional unsafe-action rate by -0.4979 ([-0.5926, -0.3686]). These shifts do not imply uniformly better behavior. Source-label SFT had the highest mean macro-F1 (0.586). The weighted arm had zero Refuted recall in every seed, and two seeds assigned NotEnoughInfo to all 167 claim examples. In an availability-amended audit at one reference seed, weighted and unfiltered outputs had 0/20 versus 1/20 evidence-supported positives and 20/20 versus 19/20 positives containing unsupported material. Samples were non-paired, source overlap was not serialized, and the amendment followed automatic summarization but preceded annotation. The audit therefore cannot estimate a common-source grounding effect and is inconclusive about system-level improvement or harm. Hard filtering already achieved 0.660 accuracy, 0.530 macro-F1, and 0.135 conditional unsafe rate. The implemented weighted arm showed no demonstrated incremental decision benefit over hard filtering. This fixed-matrix failure analysis shows decision redistribution with lost label functionality; the available human audit does not establish a grounding gain.
