Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates the efficacy of using cosine similarity to a positive response centroid as a proxy for response safety, finding it performs poorly and can be significantly outperformed by a reference direction derived from both safe and unsafe examples. Auditing this "positive-centroid rule" across multiple corpora and encoders reveals its limitations, with performance often near chance or even inverted, especially when compared to an explicitly defined safe-minus-unsafe reference. The study concludes that a class mean represents a location, not a safety direction, and that a properly oriented reference with sufficient unsafe examples is crucial for robust safety classification.
A simple "safe response centroid" embedding fails to reliably distinguish safe from unsafe LLM outputs, often performing worse than chance and requiring an explicit unsafe reference to orient the safety direction.
Can response safety be scored by cosine similarity to the mean embedding of known-safe responses? A recent sleeper-agent detector proposes exactly this score, yet the raw positive-centroid rule is not identified: positive observations locate the safe class relative to an encoder origin, but do not determine which direction separates safe from unsafe responses. We audit the rule on two prompt-controlled, human-labeled corpora and one auxiliary jury-labeled source control, using four frozen encoders and prompt-grouped splits. On the human-labeled corpora the safe prototype reaches ROC-AUC 0.457-0.545, with two cells significantly below chance and one above, while an explicit safe-minus-unsafe reference reaches 0.588-0.738 on the same embeddings; on the jury control the prototype is inverted (0.358-0.405) and the reference reaches 0.754-0.793. At validation-calibrated 5% false-safe thresholds, the reference accepts more safe responses on PKU-SafeRLHF (0.153-0.263 versus 0.039-0.061 across encoders) and Aegis (0.189-0.291 versus 0.004-0.045), but not reliably on BeaverTails. A fully unlabeled held-out reference recovers part to most of the referenced ranking, much less when only 5% of the pool is unsafe, whereas 80-634 labeled unsafe responses recover most of it. Prompt-only ablations show that prompt-label composition can inflate uncontrolled evaluations. This is a bounded result about a raw positive centroid, not all one-class methods or safety-specialized guards. A class mean is a location, not necessarily a safety direction; a declared reference with enough unsafe mass identifies orientation.