Search papers, labs, and topics across Lattice.
This paper introduces a novel framework called reasoning consistency scanning to audit the logical consistency of chain-of-thought (CoT) reasoning in AI safety evaluations. By focusing on the consistency of reasoning rather than its faithfulness, the authors provide a method that can assess transcripts without requiring experimental interventions. The findings reveal that reasoning inconsistencies are prevalent across various models and tasks, highlighting a critical area for improving AI safety evaluations.
Reasoning inconsistencies in AI outputs are not just common; they can be systematically detected and vary significantly across models and tasks.
Prior work has shown that chain-of-thought (CoT) reasoning is often unfaithful: a model's stated reasoning does not reliably reflect the process that produced its output. Detecting unfaithfulness, though, requires controlled experimental interventions, which cannot be applied to evaluation transcripts after the fact. We turn instead to a more tractable question that has received less attention: whether the stated reasoning is logically consistent with the answer it accompanies. Unlike faithfulness, consistency can be assessed from a transcript alone, with no intervention. We introduce reasoning consistency scanning, a reusable method for detecting this property in AI safety evaluation transcripts. Our contributions are fourfold. First, we formalize reasoning consistency as distinct from faithfulness and define a six-subtype taxonomy of inconsistency. Second, we build a validated benchmark of 60 transcripts, manually adapted from InstrumentalEval outputs. Third, we implement a working scanner for InspectScout, the first to target this property in safety evaluation transcripts. Fourth, we report results across four generator models and three evaluations from inspect_evals, showing that reasoning inconsistency is present, detectable, and varies systematically across both models and task types.
