Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Parallel Power Tempering (PPT), an inference-time sampling method that enhances LLM reasoning by running multiple power-sharpened replicas at different tempering levels. PPT resolves the exploration-exploitation trade-off inherent in power-sharpening by enabling lower-tempered replicas to explore diverse trajectories while higher-tempered replicas exploit promising paths. Experiments demonstrate that PPT significantly outperforms single-chain power-sharpening and RL-post-trained models, achieving reasoning quality comparable to frontier LLMs.
A novel inference-time sampling strategy unlocks frontier LLM reasoning capabilities in smaller models by resolving the exploration-exploitation dilemma.
Power-sharpened sampling is an inference-time alternative to reinforcement-learning (RL) post-training for enhancing reasoning in large language models (LLMs). High-probability sequences are amplified under the base model without parameter updates or external rewards, avoiding the costly optimization and jagged generalization of RL. However, this approach faces a fundamental exploration--exploitation trade-off, as % strong sharpening restricts exploration, trapping samplers in plausible but incorrect reasoning trajectories, whereas weak sharpening leaves the answer distribution diffuse. To resolve this trade-off, we introduce \textbf{Parallel Power Tempering (PPT)}, instantiating power-sharpened LLM sampling via parallel tempering. Running multiple \emph{interacting} replicas in parallel at different sharpening levels allows lower-power replicas to explore diverse reasoning trajectories and higher-power chains to further exploit higher-likelihood responses favored by the sharpened target. Specifically, we tailor \method{} to inference-time sampling by mitigating a truncation bias, identified in prior power samplers, and investigate effective swap strategies under finite memory and compute budgets. Extensive experimentation shows that \method{} substantially improves single-chain power-sharpened sampling and outperforms RL-post-trained models, producing higher-quality reasoning traces and even achieving performance comparable to frontier models.