Search papers, labs, and topics across Lattice.
This paper reveals that the effectiveness of runtime safety monitors based on Linear Temporal Logic (LTL) and finite state automata (FSA) is fundamentally limited by the entropy of attack distributions, with low entropy leading to high recall and high entropy resulting in near-zero recall. The authors prove that the recall of fixed-invariant FSA monitors is constrained by the concentration of attack patterns, demonstrating that architectures like GPT-class exhibit concentrated attacks while Gemini variants show high-entropy distributions. A pre-deployment entropy test is introduced, which can predict monitor coverage from a small sample of attacks, allowing for informed architecture-aware monitor selection prior to deployment.
Attack distribution entropy can predict the effectiveness of LTL-based safety monitors, revealing why some models perform poorly in safety coverage.
Runtime safety monitors based on Linear Temporal Logic (LTL) and finite automata (FSA) are increasingly deployed to intercept unsafe tool-call sequences in LLM agents. Yet the same monitor achieves 68-75% attack coverage on some model architectures and near-zero on others, with no explanation from capability scores, training data, or prompt design. We provide the missing theory. We prove that the recall of any fixed-invariant FSA monitor is bounded above by the concentration of the attack distribution: the fraction of attacks covered by the k most frequent trigger-completion patterns. When attacks concentrate (low Shannon entropy), a small fixed invariant set achieves high recall; when they disperse across many structurally distinct patterns (high entropy), no fixed invariant set of tractable size can, regardless of how the invariants were derived. We validate this entropy-coverage bound across eight frontier LLM architectures. GPT-class and DeepSeek backends yield highly concentrated attacks (H ~ 0.24 bits; one pattern covers 96%), explaining 68-75% recall; Gemini variants yield high-entropy distributions (H ~ 2.81 bits; 7 clusters each <= 7%), explaining near-zero recall (6-13%), invariant to architecture-matched retraining. Entropy accounts for 76% of variance in coverage (Pearson r = -0.87, p = 0.005, 95% CI [-0.98, -0.78]), holding under leave-one-out (r in [-0.91, -0.82]). We introduce a pre-deployment entropy test that predicts monitor coverage from a small attack sample, enabling architecture-aware monitor selection before deployment. The bound and test are architecture-agnostic and apply to any FSA-based runtime monitor over discrete action sequences.
