Search papers, labs, and topics across Lattice.
This study introduces TxBench-PP, a benchmark designed to evaluate AI agents' performance in small-molecule preclinical pharmacology by assessing their ability to derive accurate conclusions from real-world assay data. The benchmark includes 100 evaluations across various drug discovery stages, focusing on critical tasks such as mechanism-of-action reasoning and compound-target engagement. Despite testing 16 different model configurations, none of the AI systems consistently succeeded in making reliable preclinical pharmacology decisions, with the best performance yielding only 59.3% accuracy on endpoint attempts.
AI agents struggle to make reliable preclinical pharmacology decisions, with the best models achieving less than 60% accuracy in real-world evaluations.
Artificial intelligence (AI) agents promise to accelerate drug discovery by compressing interpretation and decision-making loops, but practical deployment requires trusted evaluation on realistic program decisions. We introduce TherapeuticsBench Preclinical Pharmacology (TxBench-PP), a verifiable benchmark for small-molecule preclinical pharmacology and the first focused slice of a broader TherapeuticsBench effort across drug-discovery stages and therapeutic modalities. TxBench-PP tests whether agents can recover accurate conclusions from real-world assay data rather than memorized facts from literature. The benchmark contains 100 evaluations indexed by program stage, assay type, and task structure, spanning mechanism-of-action (MoA) and pharmacodynamic (PD) reasoning, compound-target engagement, causal target validation, developability and safety, and translational efficacy. Agents receive realistic workflow snapshots, inspect files in a coding environment, and return structured answers graded deterministically. Across 16 model-harness configurations, comprising 11 models and 4,800 trajectories, no system reliably recovered preclinical pharmacology decisions. The strongest configuration, Claude Opus 4.8 / Pi, passed 59.3\% of endpoint attempts (178/300; 95\% CI, 51.1-67.6), followed by GPT-5.5 / Pi at 55.3\% (166/300; 47.0-63.6).
