Search papers, labs, and topics across Lattice.
This paper introduces SingProbe, a lightweight intrinsic runtime guard for large language models (LLMs) that leverages hidden states during inference to predict query intent, response safety, and hallucination risk with minimal overhead. By integrating these predictions directly into the autoregressive decoding process, SingProbe offers a more efficient alternative to existing external guardrails, achieving competitive performance with only approximately 2 million parameters and less than 0.5% additional inference cost. The approach not only enhances safety monitoring but also enables proactive risk management during generation, particularly in sensitive applications like medical generation through its extension, SingProbe-Med.
SingProbe achieves superior safety monitoring with negligible overhead by reusing LLM hidden states, challenging the need for bulky external guardrails.
Runtime guardrails are essential for reliable large language model (LLM) deployment, yet existing approaches typically rely on independent, external models that introduce additional inference cost, delayed safety signals, and a capacity mismatch with increasingly capable base models. To address these issues, we introduce SingProbe, a lightweight intrinsic runtime guard that directly reuses hidden states produced during LLM inference and operates alongside autoregressive decoding. Within a unified framework, SingProbe continuously predicts query intent, response safety, and hallucination risk at the token level with negligible additional guardrail inference overhead, offering a "free-lunch" solution. We further introduce SingStreamBench, a benchmark designed to assess whether streaming guardrails remain inactive on benign prefixes while promptly detecting emerging unsafe content. Extensive experiments show that SingProbe achieves competitive or superior performance compared with substantially larger standalone guardrails and specialized hallucination detectors, with only $\approx$2M parameters and $<0.5\%$ extra overhead. Beyond passive detection, we also show that SingProbe scores can anticipate future generation risk and guide constrained safe decoding. We further extend this paradigm to medical generation through SingProbe-Med, which selectively activates risk-directed decoding interventions only when clinically relevant risks emerge. Together, these results demonstrate that internal model representations provide an effective and efficient interface for generation-time monitoring and control.
