Biqing Qi

NCP-ArchPreview is introduced, a latent-space language model that pushes autoregressive pretraining beyond standard next-token prediction (NTP) and learns through Next Concept Prediction (NCP) to predict discrete concepts that span multiple tokens, introducing an explicit and more challenging concept-level objective while preserving standard token-level autoregressive generation.

Achieving similar performance to larger models with significantly less data and faster inference speeds could redefine efficiency benchmarks in foundation models.

PI-Mem achieves unprecedented long-context reasoning capabilities, outperforming traditional methods while accelerating inference by over 6 times.