Search papers, labs, and topics across Lattice.
Affiliation:
2
0
4
8
SpectralShift is proposed, a spectral reparameterization approach for long-context continual pretraining of GDNs that reparameterizes the alpha projections initialization to reshape the decay spectrum by enhancing slow propagation capacity, and further introduces a learning-rate scaling for alpha projections to facilitate long-context training.
Scaling zero RL to a trillion parameters reveals that models can spontaneously develop advanced cognitive behaviors, making traditional heuristics obsolete.
