Search papers, labs, and topics across Lattice.
Affiliation:
2
0
4
3
Experiments show that MoRE consistently achieves lower perplexity and stronger downstream performance than standard MoEs and state-of-the-art weight-sharing architectures at matched compute and parameter budgets, with only minimal modifications to existing MoE implementations.
A 5-step sampling schedule can deliver nearly the same quality as a 50-step one, slashing inference costs by 90%.
