Search papers, labs, and topics across Lattice.
Indian Institute of Technology Kharagpur (IIT KGP)
2
0
3
CWGD-Cosine can halve the optimization error floor compared to standard methods, revolutionizing how we approach learning rate schedules in SGD.
Linear attention models can mimic quadratic attention in-context learning for simple tasks like linear regression, but with limitations that this paper elucidates.
