Search papers, labs, and topics across Lattice.
This paper investigates the phenomenon of thinking-induced hallucination in large reasoning models (LRMs) during factual question answering, where explicit thinking can sometimes lead to incorrect answers. The authors introduce MARGO, a mixed-mode advantage regularization framework that leverages both thinking and non-thinking trajectories to enhance factual reliability while mitigating the risks of hallucination. Experimental results across various benchmarks reveal that MARGO significantly improves factual accuracy compared to strong baselines, without sacrificing general reasoning capabilities.
Explicit thinking in large reasoning models can both enhance and undermine factual accuracy, but MARGO effectively curbs hallucination while preserving reasoning skills.
Large reasoning models (LRMs) improve language model capabilities by generating explicit thinking traces before final answers. In factuality-oriented question answering (QA), such thinking often improves overall performance by helping the model recover relevant knowledge and refine its answers. However, we find that this benefit is not uniform at the instance level: explicit thinking can also overturn correct non-thinking answers and lead to factual drift. We refer to this failure mode as \emph{thinking-induced hallucination}. To explain this phenomenon, we formulate explicit thinking in factuality QA as a thinking residual over the model's direct-answer tendency, which can either recover missing knowledge or introduce unsupported associations. Based on this formulation, we propose MARGO, \underline{\textit{M}}ixed-Mode \underline{\textit{A}}dvantage \underline{\textit{R}}egularization for \underline{\textit{G}}rounded \underline{\textit{O}}ptimization, a reinforcement learning framework that uses non-thinking rollouts as same-model references in advantage estimation. By constructing mixed-mode rollout groups with both thinking and non-thinking trajectories, MARGO evaluates whether explicit thinking adds factual value beyond direct answering, thereby suppressing hallucination-prone thinking while preserving beneficial thinking behaviors. Experiments across multiple factuality-oriented QA benchmarks demonstrate that MARGO improves factual reliability over strong baselines, while evaluations on mathematical benchmarks show that it preserves general reasoning ability.
