Ning Ding

Value Flattening is identified as an important yet overlooked failure mode of critic learning in standard PPO and a simple sparse supervision strategy can mitigate it; SParse Proximal Policy Optimization is introduced, which applies the value loss to only a few well-separated states in each response to mitigate both effects.

On-policy distillation is massively data-overfed: a single training prompt recovers most full-dataset performance gains, while just 16 prompts saturate 98.9% of reachable state space to match full-data distillation.

Safety in AI can now evolve from within the model itself, drastically enhancing long-term interaction capabilities without external constraints.

Achieving similar performance to larger models with significantly less data and faster inference speeds could redefine efficiency benchmarks in foundation models.

Short-context models can achieve superior reasoning performance by leveraging long-context teacher models through innovative token alignment and training strategies.

Frontis-MA1 achieves a remarkable 71.21% Medal Average on MLE-Bench Lite, outperforming leading models and showcasing the potential of AI systems to recursively improve their own engineering processes.