Search papers, labs, and topics across Lattice.
We introduce RAG-PIBench, a leakage-aware benchmark with 4,876 contextual examples for evaluating prompt-injection detection in RAG systems. The benchmark's construction pipeline and strict evaluation protocol allow for a rigorous comparison of various detection methods. DistilBERT achieved state-of-the-art performance (F1 = 0.896, PR-AUC = 0.968) on the protected test set, highlighting the efficacy of transformer-based approaches.
A new benchmark reveals that even sophisticated RAG systems struggle with prompt injection, with DistilBERT leading detection at 89.6% F1.
Retrieval-Augmented Generation (RAG) systems are vulnerable to prompt-injection attacks embedded in retrieved content. We introduce RAG-PIBench, a benchmark for RAG-style prompt-injection detection containing 4,876 contextual examples across frozen train, validation, and protected-test splits. Using a leakage-aware construction pipeline and strict evaluation protocol, we compare keyword-based, semantic-reference, TF-IDF, and transformer-based detectors. DistilBERT achieves the best protected-test performance (F1 = 0.896, PR-AUC = 0.968), while TF-IDF SVM and logistic regression remain competitive. Our results demonstrate the value of leakage-aware benchmark design and strong sparse baselines for reliable prompt-injection detection in RAG systems.
