Kien Do

Black-box knowledge distillation can be significantly improved by synthesizing diverse image priors and using contrastive learning to enhance the distinctions between synthetic samples.

Augmenting few-shot knowledge distillation with adaptively selected, teacher-confident GAN-generated images dramatically boosts student accuracy.

FrameDiT achieves state-of-the-art video generation by ditching token-level attention for a novel matrix-based attention that operates directly on entire frames.