Search papers, labs, and topics across Lattice.
3
0
7
3
Experiments show that MoRE consistently achieves lower perplexity and stronger downstream performance than standard MoEs and state-of-the-art weight-sharing architectures at matched compute and parameter budgets, with only minimal modifications to existing MoE implementations.
A 5-step sampling schedule can deliver nearly the same quality as a 50-step one, slashing inference costs by 90%.
Forget expensive human annotations or hallucination-prone LLM-generated data: rule-generated synthetic data can teach LLMs to compose knowledge and significantly boost multi-hop reasoning on real-world tasks.
