Search papers, labs, and topics across Lattice.
This study evaluates the confidence levels of prompt-injection detectors, specifically ProtectAI-v2 and two Prompt-Guard-2 checkpoints, under varying attack distributions. The findings reveal that while these detectors maintain a high severity metric (0.99 to 1.00) on missed attacks, their false-negative rates can be alarmingly high (0.01 to 0.97), indicating that when they fail, they do so with near certainty. Notably, all three detectors exhibit a significant blind spot for indirect behavior-hijack injections, which could lead to severe security vulnerabilities in deployed systems.
Detectors confidently miss dangerous prompt injections, exposing a critical blind spot that could compromise system integrity.
Prompt-injection detectors are deployed as guards: a model scores an input and a downstream system trusts or blocks it on that score. I study the confidence of these scores, not only their accuracy, when the attack distribution shifts away from the clean benchmark on which the operating point was chosen. I evaluate three released detectors, ProtectAI-v2 and two Prompt-Guard-2 checkpoints, at a single source-calibrated threshold that I freeze and transport across five shifts. I report a severity metric S, how confident a detector is on the attacks it misses, alongside the false-negative rate and discrimination. Across every shift and every detector, severity on the missed attacks stays between 0.99 and 1.00 while the false-negative rate ranges from 0.01 to 0.97: when these detectors miss, they miss with near-certainty. All three confidently pass indirect behavior-hijack injection, a blind spot unanimous across two vendors and a fourfold size range. Standard pooled calibration error does not register this; one detector it rates well-calibrated, at 0.06, is miscalibrated at 0.91 on the attacks alone. Run against live models, the missed injections leak the majority of working exploits, passing them at the rate they catch others. A controlled experiment traces the cause to content-keying rather than injection structure, an instruction-tuned model used as a judge shows the same hijack blind spot, and a black-box rewriter exploits the content-keying to manufacture working confident misses, most effectively on the most dangerous attack category. Code and data are public.
