Data Curation & Synthetic Data - Weekly Roundup

Packed trajectories, not sequential rollouts, unlock high-fidelity multi-turn image editing by preventing visual drift.

Synthesizing data environments from explicit world specifications, not just files, unlocks scalable training for data agents and reveals a fundamental computation-insight gap in frontier models.

A foundation model now surpasses the Cox model on survival analysis, even for small clinical cohorts, by predicting survival curves in-context.

A 2.4B VLA model achieves state-of-the-art performance by integrating a novel register-based memory into its action expert, drastically reducing oscillations and improving real-world robotic task success.

LLMs' financial forecasts are more swayed by commentary tone and momentum than by the actual earnings surprise itself.

Packed trajectories, not sequential rollouts, unlock high-fidelity multi-turn image editing by preventing visual drift.

Synthesizing data environments from explicit world specifications, not just files, unlocks scalable training for data agents and reveals a fundamental computation-insight gap in frontier models.

A foundation model now surpasses the Cox model on survival analysis, even for small clinical cohorts, by predicting survival curves in-context.

A 2.4B VLA model achieves state-of-the-art performance by integrating a novel register-based memory into its action expert, drastically reducing oscillations and improving real-world robotic task success.

LLMs' financial forecasts are more swayed by commentary tone and momentum than by the actual earnings surprise itself.

A handful of chemically diverse unlabeled molecules can boost QSAR model performance more than massive, randomly sampled datasets.

Hundreds of billions of midtraining tokens on a curated software engineering corpus unlock substantial gains in agentic capabilities, outperforming standard SFT alone.

LLM agents can learn to proactively build optimized, hierarchical file system workspaces that anticipate future queries, outperforming reactive retrieval by 33 points on BrowseComp-Plus.

In-context learning emerges significantly earlier than specialized prediction, revealing a fundamental trade-off in how LLMs adapt to novel data based on pretraining scale.

Generative models can achieve significantly higher output diversity and corpus alignment by decoupling sampling from rendering at test time.

Robot policies learn 4x faster by co-evolving their training environments with a critic that diagnoses and recreates failure modes.

Generative representation learning unlocks compute-optimal latent tokenization, achieving superior image quality with drastically fewer FLOPs than prior methods.

Standard imputation methods fail to predict downstream utility, but task-aware imputation recovers complete-data performance in up to 80% of challenging train-test mismatch scenarios.

Foundation models trained on synthetic order flow perform amortized Bayesian inference, decoding latent market regimes with surprising accuracy and outperforming traditional methods for simulator calibration.

Exhaustive canonical skeletons for symbolic regression reveal that current state-of-the-art methods struggle on a benchmark designed to eliminate structural redundancy and simplicity bias.

Synthetic experience unlocks substantial game control improvements for LLMs, pushing success rates on complex benchmarks like MineDojo-300 from 12.63% to over 38%.

SAEs can achieve near-optimal reconstruction with orders of magnitude less data than typically used, but recovering specific, reproducible features demands significantly more.

Terminal agents can learn to overcome their own mistakes, achieving a 14.6-point Pass@1 improvement on complex coding benchmarks by recovering from failures.

Pretraining chemical language models yields diminishing returns, often underperforming classical baselines and even random initialization, suggesting current approaches capture little true chemistry.

Long-horizon agents can now achieve GPT-6 Astra and Claude Opus-level performance on complex tasks by learning from dense, progressive rewards derived from latent world specifications.

Quality-aware pretraining strategies, particularly approach-aware episode conditioning, can significantly outperform plain behavioral cloning in VLA tasks, with optimal choices depending on label granularity and model architecture.

A single dataset now unifies 279,166 multi-sensor, multi-temporal Earth observation tiles, unlocking efficient pretraining for generalist foundation models.

DNN-based IDSs trained in realistic cyber environments exhibit near-perfect in-domain detection but suffer catastrophic performance degradation with minor domain shifts.

LLM agents can now dynamically design their own post-training recipe spaces, leading to a 2.6x performance boost on complex multi-domain tasks.

Dependent sampling doesn't break KMM risk estimation; effective sample sizes derived from mixing properties provide robust finite-sample guarantees.

A 2.5B Tamil-first LLM achieves state-of-the-art translation and native language understanding, while demonstrating functional tool-calling, offering a blueprint for under-represented language AI development.

Most industrial anomaly detectors collapse under simple brightness or zoom changes, losing up to 34 AUROC points, but a new approach using synthetic anomalies maintains over 92% AUROC.