Search papers, labs, and topics across Lattice.
This paper critically evaluates the effectiveness of black-box multimodal guardrails in AI systems, revealing that the safety metrics attributed to these guardrails are often misleading due to unmeasured contributions from both the guardrail and the target model. By analyzing the interaction between different input channels and the internal processing of requests, the authors demonstrate that the guardrails can either block or allow harmful outputs based on the nature of the input and the encoding used. The findings indicate that the perceived effectiveness of guardrails can be significantly inflated, with actual contributions varying widely depending on the context, challenging the reliability of current evaluation metrics in safety assessments.
Guardrails in AI systems may claim safety benefits that are largely illusory, with their actual effectiveness hinging on the specifics of input encoding and channel dynamics.
A black-box guardrail is evaluated as though the safety number it earns were its own. It is not. A defended pipeline holds two components that can refuse (the guardrail, and the target model out of its own alignment), and every reported metric is a sum over both. We show that the guardrail's actual share of the safety credited to it runs from none of it to essentially all of it, decided by two variables no evaluation records: which channel carries the payload, and what text the harness places in the defense's internal read. The split is recoverable at no extra cost, because a guard block replaces the model's response and the two counts are therefore disjoint. On a text guard across two open-weight targets: with the payload rendered as pixels the guard blocks nothing and the model produces every refusal the system makes; reading the encoded prompt the attacker actually sent, the guard produces a minority of the refusals attributed to it; reading the unencoded request behind the attack, it blocks almost everything and the model falls silent. The blindness is not inaccuracy: the same guards block no benign image inputs either, so their image-channel decision is a constant. Granting the unencoded request inflates measured benefit substantially for a guard gate, less for a caption-mediated re-check, and not at all for a majority-vote smoother; the ordering reproduces in an independent replicate. Isolating the grant within one defense shows it does not improve detection: the harm-verdict stage contributes nothing, while the stage that regenerates the answer carries the effect. Nor is the inflated setting careless; the reference implementation builds every stage from a single prompt field that cannot distinguish what the attacker sent from what the benchmark records, so faithful porting supplies it silently. Previously published figures of our own are among those revised.
