Search papers, labs, and topics across Lattice.
This paper introduces a training-free approach to multi-step theorem prediction using in-context learning (ICL) with large language models (LLMs). It identifies and addresses the "Structural Drift" problem in deep reasoning by incorporating Theorem Precedence Graphs, which encode temporal dependencies from historical solutions to constrain the search space. The proposed method, combining retrieval-augmented graph construction and a symbolic executor, achieves 89.29% accuracy on the FormalGeo7k benchmark, matching state-of-the-art supervised models without any gradient-based optimization.
LLMs can match SOTA supervised theorem provers without training, if you give them the right structural scaffolding.
Multi-step theorem prediction is a central challenge in automated reasoning. Existing neural-symbolic approaches rely heavily on supervised parametric models, which exhibit limited generalization to evolving theorem libraries. In this work, we explore training-free theorem prediction through the lens of in-context learning (ICL). We identify a critical scalability bottleneck, termed Structural Drift: as reasoning depth increases, the performance of vanilla ICL degrades sharply, often collapsing to near zero. We attribute this failure to the LLM's inability to recover latent topological dependencies, leading to unstructured exploration. To address this issue, we propose Theorem Precedence Graphs, which encode temporal dependencies from historical solution traces as directed graphs, and impose explicit topological constraints that effectively prune the search space during inference. Coupled with retrieval-augmented graph construction and a stepwise symbolic executor, our approach enables LLMs to act as structured planners without any gradient-based optimization. Experiments on the FormalGeo7k benchmark show that our method achieves 89.29% accuracy, substantially outperforming ICL baselines and matching state-of-the-art supervised models. These results indicate that explicit structural priors offer a promising direction for scaling LLM-based symbolic reasoning.
