Search papers, labs, and topics across Lattice.
This paper investigates whether LLMs possess a shared logical subspace between natural language and symbolic representations of reasoning. They use Canonical Correlation Analysis (CCA) to identify a low-dimensional subspace within the residual activations of paired natural language and symbolic reasoning chains that maximizes cross-view correlation. By steering the LLM's reasoning chain along this learned subspace, they achieve improved logical reasoning accuracy by up to 11 percentage points across four benchmarks, demonstrating effective knowledge transfer between reasoning modalities.
LLMs can be steered to better logical reasoning by exploiting a shared, low-dimensional "logical subspace" learned by aligning natural language and symbolic reasoning views.
Large Language Models (LLMs) still struggle with multi-step logical reasoning. Existing approaches either purely refine the reasoning chain in natural language form or attach a symbolic solver as an external module. In this work, we instead ask whether LLMs contain a shared internal logical subspace that simultaneously aligns natural-language and symbolic-language views of the reasoning process. Our hypothesis is that this logical subspace captures logical reasoning capabilities in LLMs that are shared across views while remaining independent of surface forms. To verify this, we employ Canonical Correlation Analysis on the paired residual activations from natural-language and symbolic-language reasoning chains, learning a low-dimensional subspace with maximum cross-view correlation. Furthermore, we design a training-free approach that steers LLMs reasoning chain along this logical subspace, thereby leveraging the complementary reasoning signals from both views. Experiments on four logical reasoning benchmarks demonstrate the effectiveness of our approach, improving accuracy by up to 11 percentage points and generalizing well on out-of-domain problems.
