Search papers, labs, and topics across Lattice.
2
0
6
This work addresses the adverse interactions between sparsity and data repetition, and presents evidence for the core mechanisms of overfitting and its potential remediation, and suggests promising avenues for future methods to reduce over-specialization in model parameters by disrupting memorization patterns.
Switch Distillation not only enhances reasoning performance by up to 71% but also preserves factual recall, challenging the conventional trade-off in knowledge distillation.
