Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
Lifted State Policy Optimization (LSPO) learns from the local changes that improve reasoning states by defining a finite, cost-adjusted transition as the learning unit, augmented by learned continuous coordinates for textual answers within a lifted state. An energy function evaluates and orders these states, guiding a shaped reward to descend energy while accounting for edit and step costs, with an internalization loss incorporating low-energy answers into the generator. On Qwen3.5-27B, LSPO achieves aggregate accuracy matching test-time reflection across diverse benchmarks with significantly reduced latency and token usage.
LLM reasoning can be optimized by learning from the geometry of state transitions, achieving reflection-level accuracy with 28% lower latency and 25.8% fewer tokens.
Preference optimization ranks final answers and trajectory RL maximizes aggregate returns, yet neither credits the local change that turns a flawed reasoning state into a better one. We introduce Lifted State Policy Optimization (LSPO), which establishes this finite, cost-adjusted transition as the primary unit of learning. Each textual answer is augmented with a learned continuous coordinate, forming a lifted state that an energy function evaluates and orders. A shaped reward drives energy descent net of edit and step costs, and an internalization loss absorbs selected low-energy answers into the direct generator. On Qwen3.5-27B, the resulting single-pass policy reaches 71.8% aggregate accuracy across mathematics, science, code, and logic benchmarks, matching test-time reflection at 71.9% with 28% lower latency and 25.8% fewer tokens. The pattern holds across dense and mixture-of-experts backbones, and matched controls confirm that transition credit and the lifted coordinate each contribute, as does answer internalization. Our code is available at https://github.com/Venomrko/lspo.