Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
CellChain, a new benchmark with 590 cases and 2,360 evidence-condition instances, evaluates foundation models on cross-scale biological reasoning for cellular perturbation responses by linking experimental data to a provenance-aware evidence graph. The benchmark systematically varies evidence coverage, composition, and assignment to assess how model predictions depend on biological context, revealing that broader evidence coverage does not consistently improve accuracy and that matched reference histories significantly outperform swapped ones in drug-sensitivity ranking. Decision-transition analysis further highlights the sensitivity of model predictions to evidence changes, even when net accuracy remains stable.
Foundation models struggle to leverage expanded biological context for cellular perturbation reasoning, with accuracy gains often negated by evidence composition or assignment shifts.
Recent advances in foundation models and large-scale cellular perturbation datasets have driven growing interest in using LLMs to predict and reason about cellular responses. Biological reasoning in this setting involves relating molecular mechanisms, cellular responses, and functional outcomes across scales. Evaluating foundation models therefore requires assessing not only whether their predictions are correct, but also how those predictions depend on the biological evidence provided. We introduce CellChain, an experimentally grounded benchmark comprising 590 biological cases and 2,360 matched evidence-condition instances across gene-dependency prediction, cytokine-axis identification, and drug-sensitivity ranking. CellChain links perturbation experiments and external biological resources through a provenance-aware evidence graph spanning molecular, pathway, transcriptional, and phenotypic information. Task-specific conditions vary evidence coverage, composition, or assignment while preserving questions, candidate answers, reference targets, and scoring rules. Across 18 foundation models, the highest accuracies under prespecified target-evidence conditions are 57.5%, 75.5%, and 70.8%, respectively, and broader evidence coverage does not consistently improve accuracy. On the drug-sensitivity benchmark cohort, matched reference histories outperform swapped histories by an average of 13.0 percentage points in cluster-weighted accuracy. Decision-transition analysis separates corrections, newly introduced errors, and wrong-answer switches, revealing substantial answer changes hidden by small net accuracy differences. CellChain provides controlled comparisons for attributing changes in model predictions to prespecified expansions of biological context, including changes in evidence coverage, composition, and biological assignment.