Search papers, labs, and topics across Lattice.
3
0
5
3
Looping the middle layers of MoE Transformers can save up to 18% in training FLOPs while boosting performance beyond validation loss predictions.
Achieving near state-of-the-art performance for under $7K opens the door for cost-effective language model training accessible to the broader research community.
Resource-constrained LLM pretraining gets a boost with PCMind-2.1-Kaiyuan-2B's open-source release and novel techniques for data mixing, repetition, and curriculum learning.
