Search papers, labs, and topics across Lattice.
To address the "prediction-claim gap" where agent-designed scientific models achieve high benchmark scores without actually utilizing intervention features, the authors establish CELLAUDIT to audit whether input variables are computationally reachable, functionally impactful, and genuinely predictive. This intervention-use verification is critical for autonomous discovery agents that rely on metric-driven feedback and frequently overfit to baseline control profiles. Auditing agent-selected cell perturbation models revealed that a top-performing predictor (0.3153 PCC) was completely invariant to compound identity due to a singleton attention bug, while integrating falsification feedback into the discovery loop successfully restored genuine perturbation dependence across multiple benchmarks.
Autonomous scientific discovery agents can achieve state-of-the-art benchmark scores by designing models that silently ignore the biological intervention entirely鈥攔elying instead on control-profile shortcuts that mask critical architectural bugs.
AI virtual cells aim to predict cellular responses to specified interventions, yet held-out predictive performance alone does not establish use of the supplied perturbation information. This prediction-claim gap matters in agentic model discovery, where language-model agents generate and revise predictors using score-based feedback. We introduce CELLAUDIT, which audits input-use claims by asking whether an input can enter the cited computation, whether fitted predictions depend on it, and whether that dependence improves prediction of observed response. On a paired morphology-transcriptomics perturbation benchmark (BBBC047), an agent-selected predictor attains a mean held-out Global Pearson correlation coefficient (PCC) of 0.3153 but remains invariant to compound replacement; a control-profile-only predictor reaches 0.3142. Source inspection identifies a compound-query pathway blocked by singleton key-value attention, and the invariance persists after refitting with disjoint control wells. In a stratified audit of 48 candidates across two linked tasks, 47 change predictions under compound replacement on both held-out folds, but only 20 show target-loss gains with intervals above zero on both folds. On BBBC047, falsification-guided revisions recover positive mean compound contributions while retaining gains over the control-profile-only baseline. In matched sci-Plex searches, audit-enriched feedback yields higher held-out performance and larger mean compound and dose contributions across five trajectories, although paired intervals span zero. Refitting fixed designs on an independently acquired cohort shows predictive generalization need not imply generalization of input-use claims: dose contribution persists, whereas support for compound identity does not. CELLAUDIT adds a falsification layer to agentic model discovery, moving from generate-score-revise toward discover-falsify-revise.
