Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
New York University Center for Data Science , NYU Shanghai
2
0
4
2
Routing robustness, not just load balancing, is the key to reliable sparse MoE scaling, with DRMoET boosting performance by 1.4% on a 10.3B parameter model.
Attention norms, computed under a RoPE geometry, pinpoint the exact tokens in retrieved documents that unlock better long-context RAG, enabling more efficient KV recomputation.