Search papers, labs, and topics across Lattice.
This paper introduces an adaptive abstention system for LLMs that dynamically adjusts safety thresholds based on contextual signals to balance safety and utility. The system employs a multi-dimensional detection architecture with five parallel detectors combined through a hierarchical cascade, optimizing for speed and precision by progressively filtering queries. Evaluations demonstrate significant reductions in false positives, especially in sensitive domains, while maintaining high safety precision and near-perfect recall, and achieving substantial latency improvements compared to non-cascaded models.
LLM safety filters can get smarter: this adaptive abstention system slashes false positives by dynamically adjusting thresholds based on context, without sacrificing safety or speed.
Large Language Models (LLMs) deployed in production environments face a fundamental safety-utility trade-off either a strict filtering mechanisms prevent harmful outputs but often block benign queries or a relaxed controls risk unsafe content generation. Conventional guardrails based on static rules or fixed confidence thresholds are typically context-insensitive and computationally expensive, resulting in high latency and degraded user experience. To address these limitations, we introduce an adaptive abstention system that dynamically adjusts safety thresholds based on real-time contextual signals such as domain and user history. The proposed framework integrates a multi-dimensional detection architecture composed of five parallel detectors, combined through a hierarchical cascade mechanism to optimize both speed and precision. The cascade design reduces unnecessary computation by progressively filtering queries, achieving substantial latency improvements compared to non-cascaded models and external guardrail systems. Extensive evaluation on mixed and domain-specific workloads demonstrates significant reductions in false positives, particularly in sensitive domains such as medical advice and creative writing. The system maintains high safety precision and near-perfect recall under strict operating modes. Overall, our context-aware abstention framework effectively balances safety and utility while preserving performance, offering a scalable solution for reliable LLM deployment.
