Search papers, labs, and topics across Lattice.
This paper shifts the evaluation of jailbreak attacks on large language models from an attacker-centric perspective to a defender-centric approach, emphasizing the safety improvements these attacks can facilitate. The authors introduce A-MESS, a framework that utilizes Shapley-based scores to identify and select effective subsets of jailbreak attacks for enhancing model safety. Their findings reveal that traditional attack success rate metrics poorly correlate with actual safety improvements, and that optimizing attack subsets can significantly bolster safety utility in real-world settings.
Evaluating jailbreak attacks as tools for enhancing model safety reveals that traditional metrics may mislead researchers about their true utility.
Jailbreak attacks on large language models are usually evaluated by attacker-centric metrics such as attack success rate (ASR), yet an attack that breaks a model is not necessarily useful for improving its safety. We propose a defender-centric view of jailbreak evaluation, where attacks are evaluated by the downstream safety improvements they enable when used as red-teaming data for safety training. Building on this view, we introduce A-MESS (Minimal Effective Attack-Subset Selection), a setting-agnostic framework for attributing and selecting jailbreak attacks from black-box subset utility observations. A-MESS estimates AttackSHAP, a Shapley-based score that attributes marginal utility to individual attacks and selects compact attack subsets under user-specified budgets via greedy or surrogate-based optimization. Across controlled utility landscapes and real LLM safety settings, we find that ASR rankings are weakly aligned with defender-centric utility, that AttackSHAP can be estimated accurately with limited utility queries, and that directly optimizing subsets yields stronger safety utility than attacker-centric or attribution-only selection. These results suggest evaluating jailbreak attacks as resources for improving safety, not only as tools for breaking models.
