Search papers, labs, and topics across Lattice.
2
0
2
Splitting models into fully decoupled, communication-free shards cuts zero-order gradient variance by roughly $1/N$, enabling backprop-free pretraining to beat standard monolithic baselines using significantly fewer perturbations.
Probe-space preconditioning compresses zero-order LLM optimization from 100,000 steps down to 70, outperforming standard backpropagation on OPT-13B while slashing training memory requirements by 90%.
