Data Curation & Synthetic Data - Weekly Roundup

Terminal agents can learn to overcome their own mistakes, achieving a 14.6-point Pass@1 improvement on complex coding benchmarks by recovering from failures.

Pretraining chemical language models yields diminishing returns, often underperforming classical baselines and even random initialization, suggesting current approaches capture little true chemistry.

Long-horizon agents can now achieve GPT-6 Astra and Claude Opus-level performance on complex tasks by learning from dense, progressive rewards derived from latent world specifications.

Quality-aware pretraining strategies, particularly approach-aware episode conditioning, can significantly outperform plain behavioral cloning in VLA tasks, with optimal choices depending on label granularity and model architecture.

A single dataset now unifies 279,166 multi-sensor, multi-temporal Earth observation tiles, unlocking efficient pretraining for generalist foundation models.

Terminal agents can learn to overcome their own mistakes, achieving a 14.6-point Pass@1 improvement on complex coding benchmarks by recovering from failures.

Pretraining chemical language models yields diminishing returns, often underperforming classical baselines and even random initialization, suggesting current approaches capture little true chemistry.

Long-horizon agents can now achieve GPT-6 Astra and Claude Opus-level performance on complex tasks by learning from dense, progressive rewards derived from latent world specifications.

Quality-aware pretraining strategies, particularly approach-aware episode conditioning, can significantly outperform plain behavioral cloning in VLA tasks, with optimal choices depending on label granularity and model architecture.

A single dataset now unifies 279,166 multi-sensor, multi-temporal Earth observation tiles, unlocking efficient pretraining for generalist foundation models.

DNN-based IDSs trained in realistic cyber environments exhibit near-perfect in-domain detection but suffer catastrophic performance degradation with minor domain shifts.

LLM agents can now dynamically design their own post-training recipe spaces, leading to a 2.6x performance boost on complex multi-domain tasks.

Dependent sampling doesn't break KMM risk estimation; effective sample sizes derived from mixing properties provide robust finite-sample guarantees.

A 2.5B Tamil-first LLM achieves state-of-the-art translation and native language understanding, while demonstrating functional tool-calling, offering a blueprint for under-represented language AI development.

Most industrial anomaly detectors collapse under simple brightness or zoom changes, losing up to 34 AUROC points, but a new approach using synthetic anomalies maintains over 92% AUROC.

Slash LLM evaluation costs by over 270x without sacrificing accuracy, revealing that even top open-weight models struggle to outperform simple baselines on AML transaction fraud detection.

Small models can now learn new skills without forgetting old ones, thanks to a novel distillation technique that leverages a larger model to guide the learning process.

One million parallel environments are insufficient without intelligent sampling: Success Guided Sampling unlocks mega-scale RL for robots by focusing training on the edge of learned capabilities.

Rare concepts in LLM fine-tuning are significantly hindered by "prior barriers" that require targeted supervision, not just more data.

A novel hierarchical decomposition into soft communities allows for scalable attributed graph generation that balances structural fidelity and downstream utility without memorizing the reference graph.

Human motion alone, when translated into a shared latent space, unlocks zero-shot generalist humanoid control for locomotion and manipulation tasks.

Out-of-distribution data can dramatically boost discrete diffusion model performance under extreme data scarcity by strategically injecting it at low noise levels.

Large observational studies can finally boost the precision of small RCTs for treatment effect estimation without sacrificing unbiasedness or making strong confounding assumptions.

Satellite imagery unlocks fine-grained socioeconomic predictions with calibrated uncertainty, even without direct ground truth.

Frontier LLMs fail to reliably identify high-value training data or accurately predict performance gains, highlighting a critical gap in their self-improvement potential.

Synthetic data alone can match real-world GAD performance, but unlocking further gains requires models that explicitly fuse topological and semantic evidence.

Human tactile gloves costing under $11 can now generate robot-ready tactile data, slashing acquisition costs by over 95% and boosting efficiency 3.5x for dexterous manipulation.

Language models can actively select replay targets to improve their own broad capability preservation, not just retain explicitly trained tasks.

Synthesized software environments unlock a 50% boost in CUA performance on unseen benchmarks, proving scalable supervision is key.

Audio-language models excel at identifying sounds but fail to grasp the crucial context of how speakers react to them.

Forget-only unlearning demands models memorize far more than standard training, challenging the premise of efficient deletion.

A 300K-sample dataset and diagnostic benchmark unlock the challenging task of generating product-preserving, narrative-driven advertisement videos.

Pretraining neural CFD surrogates across diverse domains unlocks substantial gains in zero- and few-shot generalization, drastically reducing data requirements for new applications.

Recurrent non-stationarity unlocks data reuse, but its "safe" application hinges on a bias-variance tradeoff akin to classical settings, with a novel algorithm achieving near-optimal regret.

A unified, open-source framework and a massive 3.3 TB dataset now enable rigorous ML-driven prediction of complex multiphase flows in porous media, accelerating research in critical energy and engineering applications.

Instruction-tuned models can achieve monolithic training performance on new domains via parallel, independent LoRA adapter training and parameter-free routing, eliminating costly full-model retraining.

Differentially private synthetic time-series forecasting utility hinges on forecaster-specific window assembly, not just generator quality.

Unbiased dataset pruning can be framed as a linear program, enabling efficient, label-free subset selection that outperforms geometric methods at small budgets.

Uncoded crash factors like phone use and medical episodes are vastly undercounted in police records, with narratives revealing 2x more injury crashes than coded fields.

Transformers learn translation, rotation, and scale invariance through distinct, identifiable mechanisms, paving the way for structured equivariant function learning.

Smoothness in treatment effects unlocks near-optimal policy learning in high-dimensional, low-signal digital experimentation.

Learned relational structure from dynamics fails to transfer to inverse problems, even when edge accuracy is high, due to uncaptured approximation errors.

Optimizing BO hyperparameters with BO itself reveals configurations that significantly reduce cumulative regret on GP-inferred sample paths.

AI is poised to revolutionize ARPES, moving beyond data analysis to enable fully autonomous, self-driving experimental labs.

Provably private tabular classification is now achievable with a single forward pass, outperforming baselines and drastically cutting fitting time.

A single 500-hour visual-tactile dataset boosts zero-shot contact prediction from 13% to 38% IoU and quadruples robot manipulation success rates, proving scale and consistency are paramount for embodied learning.

Generative models can now produce inorganic crystals with oxidation-state distributions mirroring real-world synthesis, significantly boosting discovery potential.

A simulation flywheel recursively bootstraps embodied data, generating richer scenes and tasks for robot pretraining and evaluation.

Inserted objects can now be made "alive" and interact realistically within videos, solving a long-standing challenge in video editing.

Deterministic dermoscopic preprocessing actually degrades performance in leakage-controlled evaluations, suggesting current automated analysis might be over-engineering for minimal gains.

Transcriptomics-specific pretraining, not model scale, unlocks state-of-the-art clinical outcome prediction from bulk RNA-seq, even for survival tasks, with a 4.2M parameter model.

Satellite images are not $n^2$ independent samples; their effective sample size plummets to $\Theta(n^2/r^2)$ due to spatial correlation, fundamentally altering generalization analysis.

Time-series foundation models, despite their promise for "what-if" analysis, exhibit attenuated dynamics and memorylessness, requiring synthetic fine-tuning to recover accurate causal inference.

Distributed MMD with Random Fourier Features achieves near-perfect correlation with expected drift magnitude on massive e-commerce data, while per-dimension KS tests falter.

LLMs can now generate novel research ideas by explicitly learning to synthesize literature through structured "anchor" specifications, outperforming prior prompting and feedback methods.

Discovering molecules with rare properties can now be an order of magnitude faster by testing complex mixtures and deconvoluting activity, slashing experimental costs from $\mathcal{O}(d)$ to $\mathcal{O}(\log d)$ or $\mathcal{O}(1)$.