Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates the post-hoc compression of reasoning traces generated by large models to enhance the efficiency of knowledge distillation. By employing two instruction-tuned models to compress traces from Qwen3.5 and gpt-oss, the authors achieve a reduction in training tokens to 12-30% of the original size, resulting in training speedups of 2.0-7.6x and shorter inference outputs of 3-19x. While compressed traces improve efficiency significantly, they do not surpass the accuracy of raw traces, highlighting a crucial trade-off between accuracy and efficiency in model training.
Compressed reasoning traces can boost training efficiency by up to 18x while retaining up to 96% of accuracy from raw traces, but they come with a trade-off in performance.
Reasoning models produce long chain-of-thought traces that are costly to distill and encourage verbose student outputs. We study post-hoc compression of such traces before knowledge distillation. Two teachers, Qwen3.5-397B-A17B and gpt-oss-120B, generate about 283k correct traces each; two instruction-tuned models then compress them to 8.6-21.0% of their original character length. Across a 48-run main grid plus seven Qwen-teacher truncation ablations, compressed traces reduce training tokens to 12-30% of raw, speed up training by 2.0-7.6x, and shorten inference outputs by 3-19x with smaller reductions under the shorter gpt-oss teacher. However, raw traces retain the highest downstream accuracy at every scale and for both teachers. A length-matched raw-trace truncation ablation shows that compression is not merely benefiting from a smaller token budget: model-compressed traces usually beat or match naive truncation, especially for smaller students, while maintaining shorter inference outputs. Overall, reasoning-trace compression offers an accuracy-efficiency trade-off rather than a free improvement: students retain up to 96% of raw-trace accuracy while gaining up to 18x higher per-token efficiency, and at the 0.8B scale under LoRA compressed traces narrow the raw-vs-compressed gap but do not exceed raw.