Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces SIREN, a novel guard model that leverages internal representations of large language models (LLMs) to detect harmful content more effectively than existing methods. By employing linear probing to identify safety neurons and an adaptive layer-weighted strategy, SIREN achieves superior performance with 250 times fewer trainable parameters compared to state-of-the-art guard models. The findings demonstrate that utilizing internal features not only enhances detection accuracy but also improves generalization and inference efficiency, paving the way for more effective safety measures in LLM applications.
Harnessing the internal states of LLMs, SIREN outperforms traditional guard models while using a fraction of the parameters, revolutionizing harmful content detection.
Guard models are widely used to detect harmful content in user prompts and LLM responses. However, state-of-the-art guard models rely solely on terminal-layer representations and overlook the rich safety-relevant features distributed across internal layers. We present SIREN, a lightweight guard model that harnesses these internal features. By identifying safety neurons via linear probing and combining them through an adaptive layer-weighted strategy, SIREN builds a harmfulness detector from LLM internals without modifying the underlying model. Our comprehensive evaluation shows that SIREN substantially outperforms state-of-the-art open-source guard models across multiple benchmarks while using 250 times fewer trainable parameters. Moreover, SIREN exhibits superior generalization to unseen benchmarks, naturally enables real-time streaming detection, and significantly improves inference efficiency compared to generative guard models. Overall, our results highlight LLM internal states as a promising foundation for practical, high-performance harmfulness detection.