Search papers, labs, and topics across Lattice.
This paper introduces a causal evaluation protocol to measure the faithfulness of LLMs to intermediate structures in schema-guided reasoning pipelines. By intervening on intermediate structures and observing whether models update their final predictions accordingly, the authors find that LLMs often fail to update predictions after intervention, indicating a lack of causal dependence. Delegating the final decision to an external tool improves faithfulness, but prompting models to prioritize the intermediate structure does not significantly close the gap.
LLMs often fail to update their final predictions after interventions on intermediate reasoning steps, suggesting that these structures function more as influential context than stable causal mediators.
Schema-guided reasoning pipelines ask LLMs to produce explicit intermediate structures -- rubrics, checklists, verification queries -- before committing to a final decision. But do these structures causally determine the output, or merely accompany it? We introduce a causal evaluation protocol that makes this directly measurable: by selecting tasks where a deterministic function maps intermediate structures to decisions, every controlled edit implies a unique correct output. Across eight models and three benchmarks, models appear self-consistent with their own intermediate structures but fail to update predictions after intervention in up to 60% of cases -- revealing that apparent faithfulness is fragile once the intermediate structure changes. When derivation of the final decision from the structure is delegated to an external tool, this fragility largely disappears; however, prompts which ask to prioritize the intermediate structure over the original input do not materially close the gap. Overall, intermediate structures in schema-guided pipelines function as influential context rather than stable causal mediators.
