Search papers, labs, and topics across Lattice.
This paper introduces AlcaTRAz, a novel prompt-level defense mechanism against jailbreak attacks on large language models that operates without requiring access to model internals. By employing rule trees to insert controlled character-level perturbations in the input text, AlcaTRAz effectively disrupts the structural patterns that jailbreaks exploit while maintaining high utility for benign queries. Evaluated across 33 models and 22 attack types, AlcaTRAz outperforms existing defenses, achieving a significant reduction in jailbreak success rates while preserving the model's performance on benign tasks.
AlcaTRAz shifts the average jailbreak response from a severe 10 to a near-refusal score of 2, all while keeping benign query performance nearly intact.
Large language models (LLMs) are vulnerable to jailbreak attacks that bypass safety alignment through carefully crafted prompts. Many existing defenses require access to model weights or internals, making them difficult to apply to black-box deployments. We propose AlcaTRAz (Anchored Tree-Rule defense Against jailbreaks), a prompt-level defense based on rule trees that operates exclusively on the input text and requires no modification or retraining of the target model. The method automatically learns a transferable transformation rule that inserts controlled character-level perturbations at selected positions, thereby disrupting structural regularities exploited by jailbreak attacks while largely preserving the model's utility on benign queries. We evaluate the proposed method across 33 open-weight models, 22 jailbreak attack types, and a benchmark of short, single-turn benign questions, comparing against three representative prompt-level baselines (Llama Guard, RA-LLM, Goal Prioritization). Among the compared defenses, AlcaTRAz achieves the best composite security and functionality score in 73.4 % of model-attack combinations and shifts the aggregate score from a modal value of 10 (maximal-severity response to the malicious request) in the undefended setting to a modal value of 2 (near-refusal) after defense, while keeping the mean benign score within 0.27 points of the undefended baseline (8.35 vs. 8.62 on a 0-10 scale). AlcaTRAz substantially reduces but does not eliminate jailbreak success: a high-severity tail remains, and we do not consider adaptive attackers, so we position it as one layer within a defense-in-depth strategy rather than a standalone guarantee.
