Search papers, labs, and topics across Lattice.
Latent chain-of-thought models compress reasoning into continuous embeddings to reduce inference overhead, but they typically suffer from representation collapse due to an absence of granular process supervision. To resolve this, the authors implement Prototype-Mediated Process Supervision (PMPS), which aligns unequal-length latent embeddings with explicit reasoning steps via learnable semantic prototypes and progressive sequential priors. Across multiple model families, PMPS reduces output token length by over 50% compared to explicit CoT while outperforming the leading latent baseline SIM-CoT by 2.08% accuracy and surpassing CoT-SFT on GPT-2.
Process supervision does not require generating explicit text: continuous latent reasoning can halve token generation costs while outperforming standard CoT through soft prototype alignment.
Latent reasoning approaches enhance token-level efficiency and robustness by replacing verbose, explicit chain-of-thought (CoT) tokens with compact continuous-space embeddings. However, existing methods lack direct process supervision over these latent embeddings, which often leads to representation collapse and uneven information distribution. To address this, we propose Prototype-Mediated Process Supervision (PMPS), which introduces learnable reasoning prototypes as semantic anchors to provide structural process-level supervision for latent reasoning. PMPS projects latent embeddings and explicit CoT embeddings into a shared prototype space, achieving many-to-many soft alignment between unequal-length representations through prototype assignment. Meanwhile, we introduce a Progressive Sequential Alignment (PSA) module to further guide training: positional priors initially encourage sequential alignment structure, then gradually relax to permit adaptive matching. Experimental results show that PMPS compresses output token length to under 50% of explicit CoT on GSM8K-Aug. Compared to leading baseline SIM-CoT, our method achieves average accuracy gains of 2.08% across different model families. On GPT-2, PMPS even surpasses CoT-SFT. On larger models and a more challenging task, PMPS consistently attains the highest accuracy among all latent reasoning methods with comparable output length.
