Search papers, labs, and topics across Lattice.
This paper investigates a real-time safety monitoring system for large language models (LLMs) that utilizes an external verifier signal to trigger alarms when unsafe outputs are detected. By calibrating the alarm threshold through risk control, the proposed method achieves competitive performance against more complex monitoring systems based on sequential hypothesis testing. The findings highlight that a straightforward approach can effectively enhance safety in LLM deployments without the need for intricate designs.
A simple real-time monitor can match the performance of complex safety systems, proving that less can be more in LLM safety.
Despite alignment training, LLMs remain prone to generating unsafe outputs at deployment time. Monitoring outputs online and raising an alarm when safety can no longer be assumed is therefore critical. We study a simple real-time monitor that turns a verifier signal from an external model into an alarm decision by thresholding, with the threshold calibrated via risk control. In experiments on mathematical reasoning and red teaming datasets, we show that this simple design is competitive with more advanced monitors based on sequential hypothesis testing.
