Search papers, labs, and topics across Lattice.
Chinese Academy of Sciences, University of Chinese Academy of Sciences
1
0
2
Learned attention allocation patterns reveal that SWA is best positioned in lower layers, challenging conventional wisdom on attention distribution in LLMs.
