Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces SeLaR, a training-free framework to improve latent reasoning in LLMs by selectively activating soft embeddings only at low-confidence steps based on an entropy gate. To prevent soft embedding collapse, they propose an entropy-aware contrastive regularization that encourages exploration of multiple latent reasoning paths. Experiments across five reasoning benchmarks show SeLaR outperforms standard CoT and other training-free methods, demonstrating improved reasoning capabilities.
LLMs can reason better without training by selectively using "soft" reasoning steps only when they're unsure, preventing overconfidence from derailing the process.
Chain-of-Thought (CoT) has become a cornerstone of reasoning in large language models, yet its effectiveness is constrained by the limited expressiveness of discrete token sampling. Recent latent reasoning approaches attempt to alleviate this limitation by replacing discrete tokens with soft embeddings (probability-weighted mixtures of token embeddings) or hidden states, but they commonly suffer from two issues: (1) global activation injects perturbations into high-confidence steps, impairing reasoning stability; and (2) soft embeddings quickly collapse toward the highest-probability token, limiting exploration of alternative trajectories. To address these challenges, we propose SeLaR (Selective Latent Reasoning), a lightweight and training-free framework. SeLaR introduces an entropy-gated mechanism that activates soft embeddings only at low-confidence steps, while preserving discrete decoding at high-confidence steps. Additionally, we propose an entropy-aware contrastive regularization that pushes soft embeddings away from the dominant (highest-probability) token's direction, encouraging sustained exploration of multiple latent reasoning paths. Experiments on five reasoning benchmarks demonstrate that SeLaR consistently outperforms standard CoT and state-of-the-art training-free methods.