Zhichen Dong

Larger datasets can reveal hidden teacher traits in student models, even from off-task data, potentially reshaping how we approach model distillation.

Expert-aligned drafting outperforms contrastive-aware methods, leading to up to 12x faster proposal paths in decoding.