Search papers, labs, and topics across Lattice.
This paper investigates the phenomenon of futile reasoning in large language models (LLMs), where models engage in computationally expensive but semantically void reasoning on tasks beyond their capabilities. Through systematic analysis, the authors identify a dominant failure mode characterized by specious reasoning that misleads users with superficially valid outputs containing subtle errors. To mitigate this issue, they introduce Capability-aligned Reinforcement Learning (CaRL), which effectively reduces futile reasoning while maintaining performance across various task difficulties by incentivizing models to refuse tasks they cannot handle.
LLMs can be trained to recognize their limitations, significantly reducing misleading outputs while preserving overall performance.
Large language models generate computationally expensive yet semantically void reasoning on beyond-capability tasks, creating risks where plausible-sounding but incorrect derivations mislead users. We characterize this \textit{futile reasoning} phenomenon through systematic analysis, revealing universal capability overreach and systematic miscalibration between capability and behavior. The dominant failure mode is specious reasoning, which outputs look superficially valid but contain subtle errors, escalating with task difficulty. To address this, we introduce \textbf{CaRL} (\textbf{Ca}pability-\textbf{a}ligned \textbf{R}einforcement \textbf{L}earning), which aligns model behavior with capability boundaries through reward shaping that incentivizes refusal over futile reasoning and hindsight refusal augmentation that converts failures into refusal supervision. Experiments demonstrate a substantial reduction in futile reasoning while preserving performance across task difficulties, effectively achieving capability-aligned behavior without sacrificing utility. \footnote{https://github.com/icip-cas/Knowing-When-to-Quit}
