Dai-Wei Chen

Smaller, cheaper models generate superior negative examples for preference distillation, significantly boosting student performance without requiring larger teacher models.

Mean-initializing new tokens in LMs creates a degenerate embedding space that cripples fine-tuning, but a simple "grounding" step can unlock significant performance gains in generative recommendation.