P. Sattigeri

Preliminary results reveal substantial differences in safety-utility calibration across models and show that failures can emerge only after several initially safe interaction steps, motivating treating agent safety as a trajectory-level property rather than a single-turn or binary success criterion.

LLMs often answer a *different* question than the one you asked, and a clever "trace inversion" technique can reveal this misalignment, leading to state-of-the-art abstention performance.