Search papers, labs, and topics across Lattice.
To resolve the training pathologies of uniform cross-entropy in mathematical reasoning鈥攚here models over-sharpen on mastered tokens and destabilize on low-confidence tokens鈥攖he authors develop Trimmed Logit-Gap SFT (TrimSFT), which restricts supervision to tokens within an intermediate confidence band. The method computes a Gaussian weight over the logit gap between the gold token and its top competitor during the standard forward pass, discarding both extremes without requiring reference models or auxiliary forward passes. Evaluated across six models from the Llama, Qwen, and DeepMath families, TrimSFT outperforms standard SFT on five of six models and achieves gains of up to +26.9 points on MATH500.
Standard SFT wastes gradient budget on tokens models either already know or cannot yet grasp; trimming supervision from both extremes yields up to a +26.9 point boost on MATH500 with zero reference-model overhead.
Supervised fine-tuning (SFT) applies a uniform cross-entropy loss to all target tokens, even though different tokens provide unequal learning signals for mathematical reasoning. This uniform treatment can over-sharpen already mastered tokens while amplifying learning pressure on uncertain, low-confidence tokens, leading to suboptimal training dynamics. We propose Trimmed Logit-Gap SFT (TrimSFT), a simple token-level reweighting method that scales the SFT loss according to the logit gap between the gold token and its strongest competitor. TrimSFT trims supervision away from both extremes: tokens already mastered (large logit gap) and tokens weakly supported by the current model (small or negative logit gap), concentrating learning within an intermediate logit-gap region between them. We instantiate this principle with a Gaussian weight centered at margin m with bandwidth {\tau}, requiring no reference model or additional forward pass. We evaluate TrimSFT on six base models from the Llama, Qwen, and DeepMath families across five mathematical reasoning benchmarks. TrimSFT consistently improves over standard SFT, achieving the best average performance on five out of six models, with gains of up to +26.9 points over SFT on MATH500. Further analyses show that the bandwidth {\tau} matters more than the exact margin location, and that half-trim variants that remove supervision pressure from only one side yield inferior trade-offs. A token-level logit-gap distribution analysis suggests that TrimSFT reshapes model confidence in a more balanced way than uniform SFT or monotonic reweighting methods. These results suggest that reasoning SFT can benefit from trimming both extremes rather than treating all tokens uniformly.
