Data Curation & Synthetic Data

Scaling tabular foundation models with massive datasets can paradoxically degrade out-of-distribution performance, even as in-distribution metrics improve.

Open-source models can now generate complex, multi-modal computer-use environments that push frontier agent capabilities and enable broad generalization without proprietary frontier models.

VLM-guided procedural generation now unlocks novel, photorealistic 3D assets and large-scale worlds with precise object placement and animation, moving beyond predefined factories.

Internet videos unlock scalable, real-world navigation data, enabling robots to achieve 94.9% success on complex language-conditioned tasks.

Training LLMs and other models is feasible and effective using a mix of human-annotated and ML-generated labels, bypassing the bottleneck of full human annotation.

Lifelong generative models can now learn new tasks without forgetting old ones by directly evolving their velocity fields, bypassing expensive trajectory simulations.

Pretraining data filtering for protein models is surprisingly brittle, with performance on targeted viral proteins easily restored via fine-tuning for under $100.

Recursive pretraining with gated mixture recurrence unlocks substantial parameter efficiency and adaptive computation for protein language models, outperforming larger models on key structural prediction tasks.

Complementary-label learning valuation breaks at low noise rates, but a new cost-margin formulation unlocks near-perfect noise detection and performance gains.

Wet-biased data pipelines can make even ideal generative precipitation models appear poorly calibrated, masking fundamental training biases and misdirecting research.

LLMs outperform specialized TSFMs and regression baselines in intervention-aware time series forecasting, yet all models exhibit a stark performance degradation when predicting responses to novel, unseen covariate futures.

Explicitly retrieving instance-level numerical knowledge during post-training unlocks significant classification performance gains for existing time series foundation models.

Local data augmentation, not just class rebalancing, is the key to improving imbalanced binary classification recall, with anchor points near the decision boundary and diffusion kernels yielding state-of-the-art results.

Class-cycling training can unlock solutions for capacity-limited networks that standard batching misses, by reorganizing internal weight dynamics.

Packed trajectories, not sequential rollouts, unlock high-fidelity multi-turn image editing by preventing visual drift.

Synthesizing data environments from explicit world specifications, not just files, unlocks scalable training for data agents and reveals a fundamental computation-insight gap in frontier models.

A foundation model now surpasses the Cox model on survival analysis, even for small clinical cohorts, by predicting survival curves in-context.

A 2.4B VLA model achieves state-of-the-art performance by integrating a novel register-based memory into its action expert, drastically reducing oscillations and improving real-world robotic task success.

LLMs' financial forecasts are more swayed by commentary tone and momentum than by the actual earnings surprise itself.

A handful of chemically diverse unlabeled molecules can boost QSAR model performance more than massive, randomly sampled datasets.

Hundreds of billions of midtraining tokens on a curated software engineering corpus unlock substantial gains in agentic capabilities, outperforming standard SFT alone.

LLM agents can learn to proactively build optimized, hierarchical file system workspaces that anticipate future queries, outperforming reactive retrieval by 33 points on BrowseComp-Plus.

In-context learning emerges significantly earlier than specialized prediction, revealing a fundamental trade-off in how LLMs adapt to novel data based on pretraining scale.

Generative models can achieve significantly higher output diversity and corpus alignment by decoupling sampling from rendering at test time.

Robot policies learn 4x faster by co-evolving their training environments with a critic that diagnoses and recreates failure modes.

Generative representation learning unlocks compute-optimal latent tokenization, achieving superior image quality with drastically fewer FLOPs than prior methods.

Standard imputation methods fail to predict downstream utility, but task-aware imputation recovers complete-data performance in up to 80% of challenging train-test mismatch scenarios.

Foundation models trained on synthetic order flow perform amortized Bayesian inference, decoding latent market regimes with surprising accuracy and outperforming traditional methods for simulator calibration.

Exhaustive canonical skeletons for symbolic regression reveal that current state-of-the-art methods struggle on a benchmark designed to eliminate structural redundancy and simplicity bias.

Synthetic experience unlocks substantial game control improvements for LLMs, pushing success rates on complex benchmarks like MineDojo-300 from 12.63% to over 38%.

SAEs can achieve near-optimal reconstruction with orders of magnitude less data than typically used, but recovering specific, reproducible features demands significantly more.

Terminal agents can learn to overcome their own mistakes, achieving a 14.6-point Pass@1 improvement on complex coding benchmarks by recovering from failures.

Pretraining chemical language models yields diminishing returns, often underperforming classical baselines and even random initialization, suggesting current approaches capture little true chemistry.

Long-horizon agents can now achieve GPT-6 Astra and Claude Opus-level performance on complex tasks by learning from dense, progressive rewards derived from latent world specifications.

Quality-aware pretraining strategies, particularly approach-aware episode conditioning, can significantly outperform plain behavioral cloning in VLA tasks, with optimal choices depending on label granularity and model architecture.

A single dataset now unifies 279,166 multi-sensor, multi-temporal Earth observation tiles, unlocking efficient pretraining for generalist foundation models.

DNN-based IDSs trained in realistic cyber environments exhibit near-perfect in-domain detection but suffer catastrophic performance degradation with minor domain shifts.

LLM agents can now dynamically design their own post-training recipe spaces, leading to a 2.6x performance boost on complex multi-domain tasks.

Dependent sampling doesn't break KMM risk estimation; effective sample sizes derived from mixing properties provide robust finite-sample guarantees.

A 2.5B Tamil-first LLM achieves state-of-the-art translation and native language understanding, while demonstrating functional tool-calling, offering a blueprint for under-represented language AI development.

Most industrial anomaly detectors collapse under simple brightness or zoom changes, losing up to 34 AUROC points, but a new approach using synthetic anomalies maintains over 92% AUROC.

Slash LLM evaluation costs by over 270x without sacrificing accuracy, revealing that even top open-weight models struggle to outperform simple baselines on AML transaction fraud detection.

Small models can now learn new skills without forgetting old ones, thanks to a novel distillation technique that leverages a larger model to guide the learning process.

One million parallel environments are insufficient without intelligent sampling: Success Guided Sampling unlocks mega-scale RL for robots by focusing training on the edge of learned capabilities.

Rare concepts in LLM fine-tuning are significantly hindered by "prior barriers" that require targeted supervision, not just more data.

A novel hierarchical decomposition into soft communities allows for scalable attributed graph generation that balances structural fidelity and downstream utility without memorizing the reference graph.

Human motion alone, when translated into a shared latent space, unlocks zero-shot generalist humanoid control for locomotion and manipulation tasks.

Out-of-distribution data can dramatically boost discrete diffusion model performance under extreme data scarcity by strategically injecting it at low noise levels.

Large observational studies can finally boost the precision of small RCTs for treatment effect estimation without sacrificing unbiasedness or making strong confounding assumptions.

Satellite imagery unlocks fine-grained socioeconomic predictions with calibrated uncertainty, even without direct ground truth.