Search papers, labs, and topics across Lattice.
This paper addresses the challenge of converting research trajectories into verifiable evidence in industrial machine-learning experiments. By introducing an evidence-grounded framework that integrates bounded verification and post-execution claim qualification, the authors ensure that generated artifacts are supported and complete, while maintaining a clear audit trail of findings. The results show that while later experimental rounds can improve outcomes, they may also lead to non-monotonic performance, highlighting the complexities of trajectory evolution in research agents.
Non-monotonic performance in experimental trajectories reveals that later rounds can sometimes undermine earlier successes, challenging conventional assumptions in industrial ML research.
Research agents increasingly conduct multi-round machine-learning experiments in industrial recommendation settings and retain the resulting trajectories to guide later decisions. Yet a completed trajectory is not automatically evidence: generated artifacts may be unsupported or incomplete, executed rounds may be invalid or confounded, and later modifications may obscure earlier findings. We study \textbf{trajectory-to-evidence conversion}, asking what a completed research process has actually established. We introduce an evidence-grounded framework that couples bounded verification of consequential artifacts with post-execution claim qualification. A context-isolated generate--verify--repair process checks artifacts for evidence violations and missing downstream requirements before release. After execution, validity and attribution checks consolidate evidence across rounds, qualify intervention-level claims as actionable repairs, diagnostic guards, or withheld findings, and preserve admitted claims as auditable records with explicit provenance and applicability boundaries. A hybrid LLM-assisted controller subsequently applies, defers, or rejects records based on available target evidence. Record audits characterize which claims survive qualification, while downstream diagnostics identify affirmative applicability judgment as a bottleneck for the tested controller. Across paper-to-target adaptations, later rounds often improve on the first, while final rounds frequently underperform an earlier best, exposing non-monotonic trajectory evolution. Candidates produced through the complete workflow also yielded positive online lifts relative to deployed baselines.
