Data Curation & Synthetic Data

Synthesizing data environments from explicit world specifications, not just files, unlocks scalable training for data agents and reveals a fundamental computation-insight gap in frontier models.

A foundation model now surpasses the Cox model on survival analysis, even for small clinical cohorts, by predicting survival curves in-context.

A 2.4B VLA model achieves state-of-the-art performance by integrating a novel register-based memory into its action expert, drastically reducing oscillations and improving real-world robotic task success.

LLMs' financial forecasts are more swayed by commentary tone and momentum than by the actual earnings surprise itself.

A handful of chemically diverse unlabeled molecules can boost QSAR model performance more than massive, randomly sampled datasets.

Hundreds of billions of midtraining tokens on a curated software engineering corpus unlock substantial gains in agentic capabilities, outperforming standard SFT alone.

LLM agents can learn to proactively build optimized, hierarchical file system workspaces that anticipate future queries, outperforming reactive retrieval by 33 points on BrowseComp-Plus.

In-context learning emerges significantly earlier than specialized prediction, revealing a fundamental trade-off in how LLMs adapt to novel data based on pretraining scale.

Generative models can achieve significantly higher output diversity and corpus alignment by decoupling sampling from rendering at test time.

Robot policies learn 4x faster by co-evolving their training environments with a critic that diagnoses and recreates failure modes.

Generative representation learning unlocks compute-optimal latent tokenization, achieving superior image quality with drastically fewer FLOPs than prior methods.

Standard imputation methods fail to predict downstream utility, but task-aware imputation recovers complete-data performance in up to 80% of challenging train-test mismatch scenarios.

Foundation models trained on synthetic order flow perform amortized Bayesian inference, decoding latent market regimes with surprising accuracy and outperforming traditional methods for simulator calibration.

Exhaustive canonical skeletons for symbolic regression reveal that current state-of-the-art methods struggle on a benchmark designed to eliminate structural redundancy and simplicity bias.

Synthetic experience unlocks substantial game control improvements for LLMs, pushing success rates on complex benchmarks like MineDojo-300 from 12.63% to over 38%.

SAEs can achieve near-optimal reconstruction with orders of magnitude less data than typically used, but recovering specific, reproducible features demands significantly more.

Terminal agents can learn to overcome their own mistakes, achieving a 14.6-point Pass@1 improvement on complex coding benchmarks by recovering from failures.

Pretraining chemical language models yields diminishing returns, often underperforming classical baselines and even random initialization, suggesting current approaches capture little true chemistry.

Long-horizon agents can now achieve GPT-6 Astra and Claude Opus-level performance on complex tasks by learning from dense, progressive rewards derived from latent world specifications.

Quality-aware pretraining strategies, particularly approach-aware episode conditioning, can significantly outperform plain behavioral cloning in VLA tasks, with optimal choices depending on label granularity and model architecture.

A single dataset now unifies 279,166 multi-sensor, multi-temporal Earth observation tiles, unlocking efficient pretraining for generalist foundation models.

DNN-based IDSs trained in realistic cyber environments exhibit near-perfect in-domain detection but suffer catastrophic performance degradation with minor domain shifts.

LLM agents can now dynamically design their own post-training recipe spaces, leading to a 2.6x performance boost on complex multi-domain tasks.

Dependent sampling doesn't break KMM risk estimation; effective sample sizes derived from mixing properties provide robust finite-sample guarantees.

A 2.5B Tamil-first LLM achieves state-of-the-art translation and native language understanding, while demonstrating functional tool-calling, offering a blueprint for under-represented language AI development.

Most industrial anomaly detectors collapse under simple brightness or zoom changes, losing up to 34 AUROC points, but a new approach using synthetic anomalies maintains over 92% AUROC.

Slash LLM evaluation costs by over 270x without sacrificing accuracy, revealing that even top open-weight models struggle to outperform simple baselines on AML transaction fraud detection.

Small models can now learn new skills without forgetting old ones, thanks to a novel distillation technique that leverages a larger model to guide the learning process.

One million parallel environments are insufficient without intelligent sampling: Success Guided Sampling unlocks mega-scale RL for robots by focusing training on the edge of learned capabilities.

Rare concepts in LLM fine-tuning are significantly hindered by "prior barriers" that require targeted supervision, not just more data.

A novel hierarchical decomposition into soft communities allows for scalable attributed graph generation that balances structural fidelity and downstream utility without memorizing the reference graph.

Human motion alone, when translated into a shared latent space, unlocks zero-shot generalist humanoid control for locomotion and manipulation tasks.

Out-of-distribution data can dramatically boost discrete diffusion model performance under extreme data scarcity by strategically injecting it at low noise levels.

Large observational studies can finally boost the precision of small RCTs for treatment effect estimation without sacrificing unbiasedness or making strong confounding assumptions.

Satellite imagery unlocks fine-grained socioeconomic predictions with calibrated uncertainty, even without direct ground truth.

Frontier LLMs fail to reliably identify high-value training data or accurately predict performance gains, highlighting a critical gap in their self-improvement potential.

Synthetic data alone can match real-world GAD performance, but unlocking further gains requires models that explicitly fuse topological and semantic evidence.

Human tactile gloves costing under $11 can now generate robot-ready tactile data, slashing acquisition costs by over 95% and boosting efficiency 3.5x for dexterous manipulation.

Language models can actively select replay targets to improve their own broad capability preservation, not just retain explicitly trained tasks.

Synthesized software environments unlock a 50% boost in CUA performance on unseen benchmarks, proving scalable supervision is key.

Audio-language models excel at identifying sounds but fail to grasp the crucial context of how speakers react to them.

Forget-only unlearning demands models memorize far more than standard training, challenging the premise of efficient deletion.

A 300K-sample dataset and diagnostic benchmark unlock the challenging task of generating product-preserving, narrative-driven advertisement videos.

Pretraining neural CFD surrogates across diverse domains unlocks substantial gains in zero- and few-shot generalization, drastically reducing data requirements for new applications.

Recurrent non-stationarity unlocks data reuse, but its "safe" application hinges on a bias-variance tradeoff akin to classical settings, with a novel algorithm achieving near-optimal regret.

A unified, open-source framework and a massive 3.3 TB dataset now enable rigorous ML-driven prediction of complex multiphase flows in porous media, accelerating research in critical energy and engineering applications.

Instruction-tuned models can achieve monolithic training performance on new domains via parallel, independent LoRA adapter training and parameter-free routing, eliminating costly full-model retraining.

Differentially private synthetic time-series forecasting utility hinges on forecaster-specific window assembly, not just generator quality.

Unbiased dataset pruning can be framed as a linear program, enabling efficient, label-free subset selection that outperforms geometric methods at small budgets.

Uncoded crash factors like phone use and medical episodes are vastly undercounted in police records, with narratives revealing 2x more injury crashes than coded fields.