Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
Reinforcement learning alignment severely suppresses the output diversity of Large Reasoning Models (LRMs), rendering standard black-box uncertainty quantification (UQ) techniques like verbalized confidence no more effective than naive repeated sampling. To counter this systemic overconfidence without access to internal logits, the authors develop Jailbreak for Uncertainty (J4U), a technique using prompt-level relaxation operators that broaden the generation distribution by approximating an optimal policy with stronger KL-regularization toward the pre-RL reference model. Across three QA benchmarks and four LRMs—including closed-source production models—J4U reduces Expected Calibration Error (ECE) by up to 5× over existing black-box UQ baselines while demonstrating statistically significant calibration improvements across six times more evaluation settings.
Jailbreaking reasoning models turns out to be an effective antidote to RL-induced overconfidence, recovering suppressed pre-alignment entropy to slash black-box calibration error by up to 5×.
While Large Reasoning Models (LRMs) excel at complex reasoning, alignment through reinforcement learning often induces systemic overconfidence. In production environments, where logits may be unavailable, robust black-box uncertainty quantification (UQ) is essential for trustworthiness and safety. Focusing on question-answering for LRMs, we show that existing black-box methods, such as paraphrase-based self-consistency and confidence verbalization, offer little to no improvement over simple repeated sampling, suggesting that alignment suppresses useful output variability. We introduce prompt-level relaxation operators that broaden the model's effective output distribution by approximating the effect of an optimal policy obtained with a stronger KL-regularization parameter, hence closer to the reference model. Theoretically, we demonstrate that relaxation improves calibration. We propose Jailbreak for Uncertainty (J4U), a jailbreak-derived technique for UQ that empirically reproduces the behavioral signatures predicted by our relaxation theory. Across 3 datasets and 4 LRMs, including a closed-source production model, J4U's improvement over repeated sampling achieves statistical significance in up to 6 times more LRM-dataset-metric settings than the strongest black-box UQ state-of-the-art baseline we evaluate, with average ECE reductions up to 5 times larger. These results provide a practical tool for UQ in black-box LRM deployment.