Search papers, labs, and topics across Lattice.
This study investigates the challenges of extreme low-bit quantization in Large Reasoning Models (LRMs), specifically focusing on the failure modes that arise during 2-bit inference. The authors demonstrate that while low-bit quantization aims to reduce decoding costs, it often leads to increased token counts due to process instabilities, resulting in significant accuracy degradation. To mitigate these issues, they introduce two controls鈥擣P16 planning and loop rescue鈥攖hat substantially enhance accuracy in reasoning tasks, achieving notable improvements in performance metrics on benchmarks like MATH-500.
Aggressive 2-bit inference can backfire, leading to longer reasoning traces and accuracy drops, but targeted recovery methods can reverse this trend dramatically.
Large Reasoning Models (LRMs) rely on long reasoning traces, making inference expensive. While low-bit quantization reduces per-token decoding cost, we show that aggressive 2-bit inference can fail to deliver end-to-end speedup because instability in the generation process inflates total token count. Instead of merely lowering answer accuracy, 2-bit quantization often produces much longer traces with repetitive loops, budget exhaustion, delayed commitment, and unclosed reasoning segments. We analyze full reasoning traces of Qwen3 reasoning models across mathematical and commonsense benchmarks and show that accuracy degradation is tightly linked to these process-level failures. To address them, we introduce two lightweight controls: FP16 planning, which gives the 2-bit model a short high-precision outline, and loop rescue, which detects repetitive traces and either commits to an earlier answer or falls back to FP16. On MATH-500, loop rescue improves Qwen3-8B accuracy from 17.2% to 74.2%, while planning plus loop rescue improves Qwen3-32B from 65.0% to 87.2%. Overall, our results show that extreme low-bit reasoning becomes practical when its failures are treated as controllable generation pathologies: with lightweight detection and selective FP16 support, 2-bit inference can recover accuracy while preserving real end-to-end speed. Our code is available at: https://github.com/brain-lab-research/quantized-reasoning.
