Honglin Lin

Chasing large token-level teacher-student gaps in agent distillation is largely misguided: local divergence fails to predict task completion, whereas unpenalized minor deviations often trigger catastrophic multi-turn failure.

LLM datasets aren't independent islands: tracing their lineage reveals hidden redundancy, benchmark contamination, and opportunities for more diverse training data.