Shi-Wei Liu

Looped Transformers can achieve tens of thousands of effective reasoning steps without performance collapse by learning to selectively retain and decay intermediate states.

Frontier LLMs fail to reliably identify high-value training data or accurately predict performance gains, highlighting a critical gap in their self-improvement potential.

Looped MoE models can scale to over 10 iterations, achieving substantial performance gains by stabilizing recurrence and diversifying expert selection.

Spectral inheritance allows for a dramatic reduction in training steps while enhancing model accuracy, challenging traditional optimization methods in RLVR.

DLMs aren't truly parallel because their training data is too sequential, but NAP shows how data curation can unlock genuine parallel decoding and boost reasoning performance.