Search papers, labs, and topics across Lattice.
Basic Research of Artificial Intelligence Laboratory (BRAIn Lab)
4
0
4
Interleaving expensive Muon spectral steps with cheap Lion sign descent not only cuts distributed training wall-clock time by 33%, but actually beats full Muon and AdamW on final perplexity.
LoRA-TSD achieves superior performance over all existing LoRA optimizers while offering the first global convergence guarantees for low-rank adaptations.
SGD's noise isn't Brownian motion, and this difference explains why neural nets diffuse along flat valleys in the loss landscape.
LoRA fine-tuning just got a memory-efficient upgrade: LoRSum matches or beats standard LoRA performance by reformulating optimization as a proximal problem and using diagonal K-FAC approximations, all without expensive SVD.
