Yu Zhao

OneRank achieves superior multi-task recommendation performance by seamlessly integrating task-specific learning within a unified Transformer framework, eliminating the traditional encoder-predictor bottleneck.

Masking just 5% of attention heads in vision-language models tanks performance on long-context tasks, revealing a surprisingly sparse and critical set of "multimodal retrieval heads" that attend to both text and images.

Multilingual MoEs can achieve best-in-class performance-to-compute ratios, even with extreme sparsity, by strategically upcycling from dense models and exhibiting structured expert activation patterns across languages.