Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
Preferred Networks, Inc.
1
0
2
15
Ditch the learning rate decay: LLMs pre-trained with a stable learning rate after warmup show superior adaptability and downstream performance after supervised fine-tuning compared to those trained with decay-based schedules.