Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces ReDiST, a student-conditioned framework for constructing and distilling reasoning traces that improve learning beyond answer-only supervision by considering learner accessibility. ReDiST revises and repairs draft traces based on student attempts and evidence, using a reader model to select optimal targets for distillation. Experiments demonstrate ReDiST's effectiveness in enhancing both source-task and cross-benchmark performance compared to existing methods.
LLM reasoning distillation can be significantly improved by tailoring trace construction to the student's learning process, not just the answer.
A reasoning trace can accompany a correct answer without providing effective supervision for a particular student. We study how to construct traces that improve learning beyond answer-only supervision by considering both their intermediate content and their accessibility to the target learner. We introduce ReDiST, a student-conditioned trace-construction and distillation framework for evidence-rich reasoning. Starting from the student's own attempts, ReDiST revises answer-correct drafts against the supplied evidence and repairs unsuccessful drafts with additional guidance from the verified answer. A frozen student-matched reader screens and ranks original and edited candidates using answer recovery and verified-answer probability, given the question, evidence, and candidate trace. We distill the selected targets while balancing trace and final-answer supervision with an auxiliary answer loss. Construction and selection occur offline, and inference uses only the distilled student. Experiments on question-answering benchmarks show that ReDiST can improve both source-task and cross-benchmark performance over answer-only, original-trace, and the evaluated trace-selection and rewriting baselines. Across supervision recipes, performance rankings can differ between source and transfer benchmarks, highlighting the importance of evaluating trace teaching utility across distributions.