Neeraj Varshney

Amazon, Arizona State University

Papers on Lattice

Total citations

Topics

h-index

Publication activitypapers/week, last 8 weeks

Research focus

Architecture Design (Transformers, SSMs, MoE) (1)Distributed Systems & Hardware (1)Scaling Laws & Emergent Abilities (1)Training Efficiency & Optimization (1)

Frequent co-authors

Chaitanya Dwivedi (1)Binxuan Huang (1)Himanshu Gupta (1)Pratik Jayarao (1)

Papers (1)

Apr 21, 2026

Amazon Science4d ago·also ASU

Expert Upcycling: Shifting the Compute-Efficient Frontier of Mixture-of-Experts

Expert upcycling lets you scale MoEs for 32% less compute by intelligently duplicating and specializing existing experts, challenging the need to train massive MoEs from scratch.

Chaitanya Dwivedi, Binxuan Huang, Himanshu Gupta +3

Architecture Design (Transformers, SSMs, MoE)Distributed Systems & Hardware Scaling Laws & Emergent Abilities+1

Search

Neeraj Varshney

Publication activitypapers/week, last 8 weeks

Research focus

Frequent co-authors

Papers (1)