Search papers, labs, and topics across Lattice.
The paper introduces kNNGuard, a novel training-free guardrail for large language models (LLMs) that leverages hidden activations to classify prompts as safe or unsafe. By utilizing a small set of 50 labeled prompts and applying multi-layer k-nearest neighbors (kNN) to fuse activation-space and embedding-space scores, kNNGuard achieves competitive F1 scores across six diverse domains. Notably, it operates 2.7 times faster than the best existing guardrails and significantly reduces the time required for domain adaptation, making it a practical solution for real-time safety in LLM applications.
kNNGuard achieves superior safety classification for LLMs without the need for fine-tuning, running 2.7x faster than existing solutions.
Large language models (LLMs) are increasingly deployed in domains requiring guardrails to detect unsafe, off-topic, or adversarial prompts. Existing guardrails predominately rely on fine-tuning to build classifiers, which often suffer from low generalization and high inference latency. We present kNNGuard, a training-free guardrail that utilizes the activation space of an off-the-shelf LLM. Given a small bank of 50 safe and unsafe prompts, kNNGuard extracts hidden activations and performs multi-layer kNN fusing activation-space and embedding-space scores for classification. Across six domains spanning topical and security prompts, kNNGuard achieves competitive or superior F1 compared to fine-tuned state-of-the-art guardrails while running 2.7x faster than the best comparable guardrail, and 10x faster than a fine-tuned safety classifier without gradient updates or fine-tuning. Domain adaptation requires only updating the labeled bank, which can be constructed in under 10 seconds and several orders of magnitude faster than established guardrails. We also analyze the impact of system prompts, layer selection, and integration into production LLM pipelines as a configurable, low-latency guardrail.
