Kaivan Kamali

Cognizant AI Lab, UT Austin & Cognizant AI Lab

Papers on Lattice

Total citations

Topics

Publication activitypapers/week, last 8 weeks

Research focus

Architecture Design (Transformers, SSMs, MoE) (1)Scaling Laws & Emergent Abilities (1)Training Efficiency & Optimization (1)

Frequent co-authors

Hormoz Shahrzad (1)Olivier Francon (1)Babak Hodjat (1)Risto Miikkulainen (1)

Papers (1)

May 27, 2026

Cognizant AI Lab3w ago·also UT Austin

Efficient Pre-Training of LLMs through Truncated SVD Layers

LLMs can now be pre-trained with significantly less compute thanks to a new method that adaptively maintains low-rank and orthonormal weight matrices.

Kaivan Kamali, Hormoz Shahrzad, Olivier Francon +2

Architecture Design (Transformers, SSMs, MoE)Scaling Laws & Emergent Abilities Training Efficiency & Optimization

Search

Kaivan Kamali

Publication activitypapers/week, last 8 weeks

Research focus

Frequent co-authors

Papers (1)