Changxin Tian

Optimal learning rate and batch size in MoEs depend directly on the expert activation ratio rather than active or total parameter counts, unlocking predictable hyperparameter transfer even at 1/64 sparsity.

Matching effective learning rates across different training setups leads to remarkably consistent loss trajectories, challenging conventional wisdom about learning rate variability.

Shrinkage Bias in E2M1 formats could be the hidden culprit behind training instability in LLMs, but uniform grids like E1M2/INT4 offer a robust solution.

Ling-2.6 and Ring-2.6 achieve unprecedented efficiency in agentic intelligence, enabling instant responses and deep reasoning at trillion-parameter scale.

Forget benchmarks - SuperValid's capability-aligned validation loss robustly predicts downstream LLM performance across architectures, scales, and training distributions.