Track trials, findings, and clinical evidence as it lands.
Search papers, labs, and topics across Lattice.
This paper introduces Dream2Discern, a novel runtime auditor for predicting terminal outcomes in multi-agent systems by learning from hard latent variants generated through implicit augmentation. By increasing the auditor's prediction loss on existing trajectories, the method creates challenging examples without requiring additional data collection or outcome verification. Dream2Discern significantly outperforms existing methods, achieving over 90% trajectory-level weighted accuracy on benchmarks like AFTraj-2K and StepShield.
A single auditor can generate its own hard training examples, turning a static dataset into a renewable source for improved failure prediction in multi-agent systems.
Reliable deployment of LLM-based multi-agent systems calls for runtime auditors that predict terminal outcomes from observed execution prefixes. These auditors typically learn from a limited corpus of trajectories with verified outcomes. Expanding this corpus requires costly execution and verification, yet broader failure coverage alone provides no explicit criterion for identifying prefixes that the current auditor finds difficult to classify. To this end, we propose Dream2Discern, a runtime auditor trained through implicit augmentation guided by its own prediction loss. During Dream, we select perturbations of the auditor's representation operators to increase prediction loss on recorded prefixes, yielding hard latent variants. During Discern, the auditor learns to predict execution outcomes from these variants using the original labels. The variants are regenerated as the auditor learns, turning a limited corpus into a renewable source of hard examples and requiring no additional trajectory collection or outcome verification. Extensive evaluations on AFTraj-2K and StepShield show that Dream2Discern outperforms competitive failure auditors and achieves over 90% trajectory-level weighted accuracy on both benchmarks while balancing prediction accuracy and timeliness. Project page: https://zbox1005.github.io/dream2discern-project/