Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces S2B-LM, a modified Symbolic Behavior Benchmark designed to isolate and measure Compositional Learning Behaviors (CLBs) in theorem provers by removing numerical confounds and incorporating chain-of-thought prompting. Evaluating ten Lean 4 provers, the study reveals a statistically significant correlation between CLB competency and performance on Olympiad-level formal mathematics problems (miniF2F > 75%). The key finding is that CLB competency is a necessary, but not sufficient, condition for achieving high performance on the most challenging formal mathematical verification tasks.
Cracking the hardest formal math problems requires more than brute-force search: compositional learning is a must-have.
Self-evolving scientific agents capable of conquering the hard tail of formal mathematics require Compositional Learning Behaviours (CLBs) -- the capacity to ground and recombine novel symbolic structures in context, beyond mere recombination of prelearned atoms. We propose \textbf{S2B-LM}, an adaptation of the Symbolic Behaviour Benchmark that removes numerical processing as a confound and adds chain-of-thought scaffolding to elicit rather than merely probe latent CLB competency. Cross-evaluating ten Lean~4 theorem provers on CLB competency (adj-ZSCT) and miniF2F whole-proof performance, exact permutation tests establish a hierarchical necessity structure: search-heavy models cover the tractable bulk without detectable CLBs, yet every model breaking into the Olympiad-level tier (miniF2F $>75\%$) is among the five highest CLB scorers ($p=0.004$). After ruling out model scale as a confound, our results show that CLB competency is \emph{necessary but not sufficient} for the hard tail of formal mathematical verification.