Search papers, labs, and topics across Lattice.
This paper introduces NeuReasoner, a theory-grounded instrument designed to explore the boundaries of reasoning elicitation in large language models (LLMs). By integrating a Neuro Lens and a Cognitive Lens, NeuReasoner evaluates cognitive tasks from CogBench alongside traditional benchmarks, revealing that while it enhances reasoning capabilities in areas like arithmetic and code generation, it struggles with decision-making under uncertainty. The findings highlight the complex interplay between model scale and elicitation, providing a clearer understanding of where reasoning recovery is effective and where it fails.
NeuReasoner reveals that while LLMs can excel in certain reasoning tasks, they still falter in critical areas like decision-making under uncertainty, challenging previous assumptions about their capabilities.
A growing body of work suggests that the reasoning capabilities of large language models are largely latent in their base form, with post-training primarily amplifying rather than introducing them. However, this evidence comes mainly from mathematical and coding benchmarks, leaving the boundary conditions of that claim largely unexplored, namely which cognitive tasks can be recovered through elicitation and where that recovery fails. To investigate this, we introduce NeuReasoner, a theory-grounded elicitation instrument. At each step, an orchestrator pairs a Neuro Lens, inspired by functional specificity, with a Cognitive Lens, drawn from the Erotetic Theory of Reasoning, and integrates their outputs through internal modularization of a single model, without external tools. We evaluate NeuReasoner on CogBench, a suite of behavioral tasks from cognitive psychology, alongside standard mathematical and coding benchmarks, measuring both its improvement over vanilla inference and its ability to match a model's post-trained thinking mode. At sufficient scale, NeuReasoner matches or exceeds thinking-mode baselines on arithmetic reasoning, code generation, Bayesian reasoning, and reward learning; these gains persist against self-consistency and iterative-refinement baselines matched to NeuReasoner's per-decision call budget. Using NeuReasoner allows us to find clear boundaries: risk-taking and decision making under uncertainty remains hard to recover through elicitation alone, and model scale interacts with elicitation in both directions: widening its advantage on some cognitive signatures while erasing it on others. Overall, through NeuReasoner as a modular, interpretable, theory-grounded elicitation instrument, we empirically map where reasoning elicitation succeeds and fails, beyond the mathematical and coding benchmarks where prior claims have rested.
ETH