Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates the faithfulness of chain-of-thought (CoT) reasoning in LLMs by extracting and manipulating monosemantic features using sparse autoencoders and activation patching. By swapping CoT-reasoning features into noCoT runs on GSM8K, the authors demonstrate a significant increase in answer log-probabilities in Pythia-2.8B, but not in Pythia-70M, indicating a scale-dependent effect. The study also reveals that CoT leads to higher activation sparsity and feature interpretability in the larger model, suggesting more modular internal computation.
Chain-of-thought reasoning isn't just window dressing: swapping CoT-related features into smaller LLMs reveals a scale threshold where it actually steers the model towards correct answers.
Chain-of-thought (CoT) prompting boosts Large Language Models accuracy on multi-step tasks, yet whether the generated"thoughts"reflect the true internal reasoning process is unresolved. We present the first feature-level causal study of CoT faithfulness. Combining sparse autoencoders with activation patching, we extract monosemantic features from Pythia-70M and Pythia-2.8B while they tackle GSM8K math problems under CoT and plain (noCoT) prompting. Swapping a small set of CoT-reasoning features into a noCoT run raises answer log-probabilities significantly in the 2.8B model, but has no reliable effect in 70M, revealing a clear scale threshold. CoT also leads to significantly higher activation sparsity and feature interpretability scores in the larger model, signalling more modular internal computation. For example, the model's confidence in generating correct answers improves from 1.2 to 4.3. We introduce patch-curves and random-feature patching baselines, showing that useful CoT information is not only present in the top-K patches but widely distributed. Overall, our results indicate that CoT can induce more interpretable internal structures in high-capacity LLMs, validating its role as a structured prompting method.