Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper critiques the use of centered token log-probability increments as a monitoring observable for low-bit quantized reasoning models, demonstrating that it inadequately assesses trajectory health due to its mean-zero martingale property. Instead, the authors introduce a training-free decoding controller that integrates a degeneration-aware alarm score with a calibrated e-process-inspired sequential detector, significantly improving detection of unreliable generations. Their approach achieves a precision of approximately 0.6 while maintaining a low false alarm rate, although it results in only a modest increase in accuracy on the GSM8K benchmark.
Centered token log-probability fails as a reliable monitor for reasoning models, but a new calibrated decoding controller enhances detection of unreliable outputs without extensive retraining.
Low-bit quantization makes small reasoning models inexpensive to deploy but can degrade their chains of thought. This motivates decoder-side monitors that intervene when generation becomes unreliable. We show that a natural candidate, the centered token log-probability increment $\log p(w_t)+H_t$, is the wrong observable for this purpose. Under the model's own sampling law it is a mean-zero martingale by construction, so it measures sampling self-consistency rather than trajectory health and is nearly silent during confident repetition, where both $\log p(w_t)$ and entropy are close to zero. We introduce a training-free decoding controller that combines (i) a degeneration-aware alarm score fusing token uncertainty with explicit verbatim repetition and (ii) a calibrated e-process-inspired sequential detector. The raw product process is Ville-valid under a conditional-mean null, while the deployed CUSUM-floored statistic is treated as an empirical change detector because the score is history-dependent and autocorrelated. On GSM8K with DeepSeek-R1-Distill-Qwen-1.5B in FP16 and INT4, calibration turns a monitor that fires on 93--95% of generations into a selective detector of failing traces ($φ\approx 0.3$, precision $\approx 0.6$ against a 0.38 base rate). In this pilot, the controller reduces measured verbatim-degeneration signals and yields a positive but statistically inconclusive INT4 accuracy change from 63% to 69% (paired McNemar $p=0.18$, $n=100$), at a 28% token-budget cost. We also find that non-termination, rather than looping, is the dominant failure mode on GSM8K. The main contribution is methodological: an explanation of why centered token log-probability is inadequate for decoder monitoring and a calibrated, cautiously evaluated replacement.