Jenia Jitsev

Frontier pretraining runs routinely stake tens of millions of dollars on scaling laws, yet the empirical pipelines used to fit and extrapolate those curves have never been systematically benchmarked until now.

This work shows that Kimi Delta Attention (KDA) can realize 2D rotations by combining a single delta-rule transformation with a second reflection supplied by its channel-wise gate, and characterize the expressivity of CKDA and prove that every orthogonal diagonal-plus-rank-one matrix is exactly a CKDA transition matrix.

Models trained on LAION-BVD achieve state-of-the-art performance in multimodal tasks, showcasing the dataset's potential to redefine video understanding.

LLMs trained on a synthetic corpus can outperform native data benchmarks while using significantly fewer tokens, challenging the assumption that more data always leads to better performance.

Training data diversity is the secret sauce that boosts agentic model performance, with OpenThoughts-Agent achieving a notable accuracy leap over existing benchmarks.