Pavel Chizhov

Collapsing pre-, mid-, and post-training into a single fully synthetic run matches open-weight model performance while slashing required training tokens by up to 140×.

Forget expensive data annotation: this pipeline generates synthetic social media data that trains models to SOTA performance, all while preserving user privacy.

Code tokenizers are wasting tokens on source-specific noise, but a simple regularization technique can prune the fat and improve efficiency without sacrificing inference speed.