Search papers, labs, and topics across Lattice.
This paper investigates the impact of extended Chain-of-Thought (CoT) on Large Reasoning Model alignment, formalizing safety degradation across reasoning depth using a novel Alignment Loss Rate (ALR) metric. The authors discover an "alignment collapse" phenomenon caused by attention dilution鈥攚here extended CoT tokens compete with and override original safety constraints鈥攅nabling a new "Reasoning Trap" jailbreak that forces long reasoning paths to amplify adversarial prompts. To counter this vulnerability, they propose Reasoning Residual Alignment (RRA), a lightweight architectural defense that dynamically re-attends to original prompt embeddings via residual connections during the reasoning trajectory.
Deeper reasoning creates an unexpected safety backdoor: extended CoT dilutes attention away from system constraints, making models progressively more vulnerable to jailbreaks the longer they "think."
The emergence of Chain-of-Thought (CoT) has established a robust foundation for Large Reasoning Models (LRMs). While deep reasoning is widely believed to enhance safety alignment, the stability of alignment mechanisms under extended reasoning remains underexplored. This paper challenges the prevailing view by revealing a critical vulnerability: Deep Reasoning May Induce Alignment Collapse. To rigorously quantify this phenomenon, we propose the Alignment Loss Rate (ALR) metric. Our experiments demonstrate that as reasoning depth increases, ALR rises significantly, indicating a severe degradation in model robustness against external perturbations. Capitalizing on this instability, a novel jailbreaking paradigm, Reasoning Trap (RT), is proposed. RT induces the model into extended reasoning to amplify the impact of adversarial attacks, leading to a sharp decline in safety capabilities. To elucidate the mechanism behind this collapse, we identify Attention Dilution as the root cause, arising from the competition for attention between the extended reasoning process and the original input. To mitigate this, Reasoning Residual Alignment (RRA), a lightweight defense strategy that dynamically re-emphasizes the input via residual connections integrated with the reasoning process.
