Search papers, labs, and topics across Lattice.
This paper introduces Reflex-Guard, a local guardrail system designed to enhance the safety of Large Language Models (LLMs) by utilizing jailbreak-aware preprocessing and dense semantic embeddings. Unlike existing solutions that incur significant latency and privacy concerns, Reflex-Guard achieves a remarkable 95.9% recall on harmful prompts with an end-to-end latency of just 37.6 ms. The approach outperforms current baselines, demonstrating superior efficiency in detecting various attack types while maintaining rapid response times crucial for real-time applications.
Reflex-Guard filters harmful prompts with 95.9% accuracy in just 37.6 ms, revolutionizing real-time safety for LLMs.
Large Language Models (LLMs) in real-world applications often face the risks of specially crafted prompts designed to bypass the safety controls. Existing guardrail methods, such as LLM-as-a-judge and cloud-based safety APIs are able to detect unsafe content. However, they often add a delay of about 250-900 ms to each request. This delay is too high for real-time applications, when the system usually needs to respond in less than 100 ms. Furthermore, routing user prompts through external moderation endpoints raises significant data privacy concerns. This paper introduces Reflex-Guard, a lightweight guardrail that runs locally. It uses jailbreak-aware preprocessing, compact sentence-transformer embeddings, and seven fast binary classifiers. Together, these components enable high-accuracy prompt safety filtering with much lower latency than existing solutions. Through systematic evaluation on a strategically balanced dataset of 30,568 samples drawn from five complementary sources, we demonstrate that Reflex-Guard achieves 95.9% recall on harmful prompts at 37.6 ms end-to-end latency. It is faster than existing baselines, including Llama Guard 2 at 255 ms and SafeDecoding at 723 ms. It can detect 100% of GCG suffix attacks and Base64-encoded prompts using the default threshold. However, DrAttack structured prompts required lowering the threshold to 0.03 for optimal detection, as they produced a distinct probability distribution. Reflex-Guard achieves Reflex Efficiency Score (RES) scores up to 16.79, significantly outperforming Llama Guard 2 (11.90) and SafeDecoding (9.80). This analysis offers practical deployment advice and shows that different attack types occupy distinct regions in the embedding probability space.
