Search papers, labs, and topics across Lattice.
This paper addresses the challenge of multi-turn jailbreaking in large language models (LLMs) by introducing a novel framework called decomposed credit GRPO (DC-GRPO) that enables precise credit assignment for each turn in a dialogue. By leveraging both immediate and future credit signals, DC-GRPO effectively mitigates the misassignment issues that arise from traditional trajectory-level scoring, leading to significant improvements in attack success rates. The method achieves impressive average ASR5@3 scores of 98.26% and 97.88% across various victim LLMs, far surpassing existing state-of-the-art approaches.
Turn-level credit assignment can boost jailbreak success rates to over 98%, revealing the limitations of traditional trajectory-based methods.
Modern large language models (LLMs) operate in interactive multi-turn settings, making multi-turn jailbreaking a realistic threat model and an important setting for automated red teaming. A core challenge in learning multi-turn jailbreak attackers is credit assignment: different turns contribute differently to the final outcome, yet existing learning signals are often too coarse to identify their individual contributions. We propose decomposed credit GRPO (DC-GRPO), a unified turn-level credit assignment framework for Group Relative Policy Optimization in multi-turn jailbreak learning. DC-GRPO assigns a separate group-relative learning signal to each turn by combining immediate and future credit, avoiding the credit misassignment induced by broadcasting a single trajectory-level score across the dialogue. We instantiate this framework with static and dynamic weighting rules that differ in how the two credit sources are balanced while sharing the same turn-level structure. Across multiple victim LLMs and benchmarks, the dynamic- and static-weighted variants achieve average ASR5@3 scores of 98.26% and 97.88%, respectively, substantially outperforming the state-of-the-art methods, including SEMA (86.58%) and TROJail (86.23%). Their consistently strong performance indicates that the central empirical benefit comes from turn-level group-relative credit assignment rather than a particular weighting rule. Warning: This paper contains examples of harmful content.
