Search papers, labs, and topics across Lattice.
EpiBench is a newly introduced benchmark designed to evaluate AI agents' capabilities in short-horizon epigenomics analysis through a series of 106 evaluations across various workflows. The study found that no AI system successfully passed the majority of attempts, with the highest-performing model, GPT-5.5 / Pi, achieving only 45% success in making accurate analysis decisions. This highlights significant limitations in current AI models' ability to perform complex, domain-specific scientific reasoning despite their proficiency in retrieving relevant data and generating intermediate results.
AI agents struggle with complex epigenomics tasks, with the best model only achieving a 45% success rate in verifiable analysis decisions.
We introduce EpiBench, a verifiable benchmark for short-horizon epigenomics analysis. EpiBench evaluates whether agents can make well-defined analysis decisions from realistic workflow states and return deterministically gradable answers. The benchmark includes 106 evaluations across CUT\&Tag/CUT\&RUN, ATAC-seq, ChIP-seq, and DNA methylation workflows. Across 5,088 valid trajectories from 16 model-harness pairs, no system passed a majority of attempts: GPT-5.5 / Pi led at 45.0\% (143/318 attempts; 95\% confidence interval (CI), 36.3--53.7), followed by GPT-5.5 / OpenAI Codex at 39.9\% (127/318 attempts; 95\% CI, 31.6--48.3). Claude Opus 4.8 Max / Pi and GPT-5.4 / Pi each passed 39.0\% (124/318 attempts; 95\% CI, 30.2--47.8 and 31.0--47.0, respectively). Performance varies across assay types, and many failed runs still contain parts of the correct answer. Agents often found the right files and computed useful intermediate results, but failed when the task required deeper, assay-specific scientific judgment.
