Mila

Grokking's distinct phases—nuisance fitting, core learning, and delayed generalization—are analytically separable, with sharp conditions for occurrence and predictable plateau lengths.

Transformer and RNN tractography models achieve state-of-the-art performance by introducing a novel generation-validation phase for streamline-level supervision, overcoming the limitations of local loss functions.

Kilometer-scale extreme heat downscaling accuracy scales linearly with climatological distance, not simulation volume, revealing a path to cost-effective urban adaptation.

Context compaction in agentic RL can run up to 5x faster simply by streaming the KV cache instead of flushing it—accidentally turning standard LLMs into recurrent agents that preserve evicted context purely through RL.

Privileged Value Functions can inject crucial token-level signals into LLM reinforcement learning, leading to substantial performance gains over traditional methods.

Latent guidance in the Reinformed Dreamer leads to more robust representation learning, outperforming previous methods in diverse benchmarks.

MOJO outperforms traditional supervised learning models by effectively utilizing unlabelled data, achieving state-of-the-art results in neural decoding tasks with minimal labelled input.

Fast transductive rates in semi-supervised learning can be achieved with fewer labels than previously thought, thanks to the power of data augmentation.

Trust-region optimization can dramatically enhance the training of neural quantum states, achieving stability and speed at unprecedented scales.

Gradual transitions in training objectives can significantly enhance model performance during adaptation, preserving valuable learned features.

Negative token filtering enables stable single-rollout training, outperforming traditional group-based methods on agentic tasks while maintaining efficiency.

MambAdapter achieves superior performance in audio and speech tasks while drastically cutting down on computational resources.

Freezing parameter groups based on Fisher structural drift can enhance fine-tuning efficiency and boost performance in downstream tasks.

SHAPO redefines safe exploration by biasing policy updates toward caution in uncertain regions, leading to significant improvements in safety and performance.

Prefix failure in on-policy distillation can be effectively mitigated by correcting problematic prefixes, leading to significant improvements in reasoning coverage and accuracy.

Language specialization in multilingual MoEs happens mostly in the final layers, suggesting a surprisingly simple recipe for parameter-efficient adaptation.

Forget expensive downstream evaluations: token-level statistics from expert-written solutions can reliably forecast LLM performance with 10,000x less compute.

Multi-modal alignment in symbolic regression models like SNIP doesn't actually improve during optimization, suggesting current approaches are too coarse to effectively guide symbolic search.

MoEs don't always need learned routers: routing information can be embedded directly in the hidden state.

Takeuchi's Information Criterion (TIC) accurately predicts DNN generalization gaps, but only when models operate near the Neural Tangent Kernel (NTK) regime.

Forget full fine-tuning: this dynamic routing strategy lets you adapt dense retrieval to new domains while using just 2% of the parameters.