Search papers, labs, and topics across Lattice.
Stanford HAIAcademyMonta Vista High SchoolSchoolUC Santa CruzUniv. HelsinkiJun 15, 2026arXiv:2606.16118This study investigates the faithfulness of Large Language Models (LLMs) in legal reasoning by comparing their performance across three paradigms: pure classification, LLM-based Formal Reasoning, and solver-based Formal Reasoning using the Z3 SMT solver. The authors find that while LLM-based Formal Reasoning achieves the highest accuracy, it does not guarantee faithful logical inference, as many legally sound inferences lack formal grounding due to unstated assumptions. The research highlights three critical failure modes, particularly scope laundering, which raises significant concerns about the reliability of LLMs as substitutes for traditional symbolic reasoning methods.
Despite high benchmark performance, LLMs often misrepresent logical reasoning, revealing a troubling gap between accuracy and faithfulness in legal contexts.
Large Language Models (LLMs) achieve strong performance on reasoning tasks, but whether this reflects faithful logical inference or heuristic approximation remains unclear. We study this question in legal entailment by comparing three paradigms, including pure LLM classification, LLM-based Formal Reasoning, and solver-based Formal Reasoning using the Z3 SMT solver, on a re-annotated subset of ContractNLI across five LLMs. Our re-annotation reveals a systematic and measurable gap between pragmatic legal interpretation and strict formal entailment, where a substantial proportion of legally sound inferences are not formally grounded without additional unstated assumptions. While introducing formal structure improves accuracy, with LLM-based Formal Reasoning achieving the highest benchmark performance, we show that this gain does not imply faithful reasoning. We identify three recurring failure modes: scope laundering, where LLMs report solver-inconsistent classifications without executing the underlying formal reasoning, producing conclusions that appear logically grounded but are not; implicit constraint blindness, where LLMs overlook logical constraints present in formal representations; and program synthesis failures, where LLMs generate incorrect Z3 code despite structured prompting. Critically, scope laundering persists across all models, raising serious concerns about the faithfulness of LLM-based formal reasoning as a proxy for symbolic execution. These results reveal a fundamental gap between benchmark accuracy and logical faithfulness.
