Search papers, labs, and topics across Lattice.
GeoFaith introduces a spatio-temporal framework leveraging latent geometric structure and entropy dynamics to diagnose and enforce faithful Chain-of-Thought (CoT) reasoning in LLMs. They train an 8B "faithfulness detector" using a bootstrapped dataset of 20k samples, outperforming GPT-4 on faithfulness benchmarks. This detector is then integrated into a reinforcement learning framework that jointly optimizes outcome correctness, process faithfulness, and trajectory consistency, leading to more faithful and interpretable CoT chains.
LLMs can be steered to generate more faithful reasoning chains without sacrificing accuracy using a novel geometric and entropy-based framework, outperforming even GPT-4 in faithfulness detection.
Chain-of-Thought (CoT) reasoning has advanced large language models (LLMs), but outcome-based supervision leads to pervasive post-hoc rationalization, producing plausible yet unfaithful reasoning chains. Most prior faithfulness assessment methods are either unscalable, expensive, or unreliable. We propose GeoFaith, a spatio-temporal framework that leverages latent geometric structure and entropy dynamics to diagnose and enforce faithful reasoning. We develop a scalable bootstrapping pipeline expanding step-level annotations from 1k to 20k samples across four domains, train an 8B faithfulness detector outperforming GPT-5 on standard benchmarks, and design a faithfulness-aware reinforcement learning framework jointly optimizing outcome correctness, process faithfulness, and trajectory consistency. Experiments show the proposed method achieves superior performance on both faithfulness detection and downstream reasoning, producing shorter, more interpretable chains without sacrificing accuracy. Our code will be made available publicly.
