Search papers, labs, and topics across Lattice.
Affiliation:
2
0
3
This work proposes MomentUm SpEctral Clipping (Musec), which replaces Muon's spectral flattening with spectral clipping: rather than setting all singular values of the momentum matrix to approximately one, Musec clips singular values that exceed a threshold while preserving the underlying spectral structure of the momentum.
AdamW's second-moment accumulator may obscure its ability to converge under heavy-tailed noise, raising critical questions about its effectiveness in training large language models.
