Search papers, labs, and topics across Lattice.
Peking University
2
0
4
Scale vectors, despite being a tiny fraction of LLM parameters, are critical for pre-training, and this paper unlocks how to make them even better with simple, theoretically-grounded tweaks.
LLMs can now scale depth more effectively: a new attention mechanism recovers diluted features in deeper layers, boosting performance with negligible overhead.
