Search papers, labs, and topics across Lattice.
School of AI, Shanghai Jiao Tong University, Dots Studio, Xiaohongshu Inc.
2
0
4
5
Multi-head attention applied to network depth unlocks substantial performance gains and compute efficiency in large MoE models.
Scaling Hyper-Connections beyond four streams is now feasible, yielding substantial performance gains without prohibitive costs.
