Search papers, labs, and topics across Lattice.
This study introduces FOCAL, a code LLM-based discriminative oracle predictor that directly predicts the pass/fail status of test prefixes rather than generating assertions. By training on labeled pairs of test prefixes and methods, FOCAL emphasizes failing cases during training and provides explanations grounded in statement-level behavioral evidence. The results show a significant performance improvement in detecting failures on unseen projects compared to the baseline method SEER, highlighting the potential of fail-aware prediction to enhance existing testing methodologies.
FOCAL outperforms traditional methods by significantly improving failure detection on unseen projects while providing rich behavioral explanations.
Despite their central role in fault detection, test oracles remain challenging to construct effectively. Recent learning based methods address this challenge by automatically generating test assertions, yet even if syntactically correct, they are often ineffective in revealing bugs. Rather than generating assertions, this study explores a different approach by training a model to directly predict whether a given test prefix passes or fails. We present FOCAL, an emerging code LLM-based discriminative oracle predictor. It learns from labeled pairs of test prefixes and methods under test, employs losses that emphasize failing cases during training, and grounds its predictions in statement level behavioral evidence. Compared with the baseline method SEER, we substantially improve performance on failing cases for unseen projects and provide richer explanations. A preliminary evaluation on fault-detection benchmarks and automated test-generation artifacts shows that our approach is highly accurate within its training distribution and substantially improves failure detection on previously unseen projects where prior discriminative oracles collapse. Moreover, the highlighted statements are supported by behavioral explanation checks. These early results suggest that fail-aware discriminative oracle prediction can complement existing approaches such as fuzzing, search-based testing, and LLM-based test generation. These techniques produce test prefixes at scale but often lack fault oriented oracles. In future work, FOCAL could take generated test prefixes and attach fault-aware predicted oracles to them, turning high-volume input generation into executable tests that are more likely to expose semantic failures.
