Search papers, labs, and topics across Lattice.
This paper introduces OpenSafeIntent, a benchmark designed to evaluate model behavior across varying intents while maintaining a fixed underlying task, allowing for a nuanced assessment of safety in AI completions. The study reveals that models often fail to consistently provide safe responses when faced with benign, dual-use, and malicious variants of the same task, highlighting significant brittleness in dual-use behavior and the unreliability of high-level answers on sensitive topics. The findings advocate for a shift in safety evaluation towards intent-calibrated assessments, emphasizing the need for models to adapt their safety measures in response to intent variations rather than relying on average performance metrics.
Models frequently misjudge safety across different intents, revealing critical vulnerabilities in AI completion systems that could lead to harmful outcomes.
Safe completion requires models to provide useful assistance without enabling harm, but this behavior is difficult to evaluate with isolated prompts. We introduce OpenSafeIntent, a benchmark of controlled prompt-sets that vary intent while holding the underlying task fixed. Each datapoint contains benign, dual-use, and malicious variants of the same task. This design lets us evaluate whether models calibrate assistance across intent shifts, rather than merely appearing safe on average. Across a broad model suite, we find that prompt-level safety hides important failures: models often fail to remain safe across matched intent variants, dual-use behavior is brittle under paraphrase, high-level answers on risky topics are not reliably safe, and responses that reframe ambiguous requests into safer tasks are substantially less likely to cross the safety boundary. Our results suggest that safe completion should be evaluated as intent-calibrated behavior over controlled task variants, not as a single safety-helpfulness tradeoff over independent prompts.
