Search papers, labs, and topics across Lattice.
MedBench v5 introduces a comprehensive benchmark for evaluating clinical multimodal models by shifting from static question-answering to a dynamic, process-oriented framework. It employs a dual-dimensional approach that assesses Clinical Cognitive Responsiveness and Medical Atomic Skills across 63 tasks, while integrating stressors to analyze model performance under various degradation scenarios. Experiments reveal that high task performance does not equate to stable processes, highlighting vulnerabilities in contradiction detection and hallucination management, which are critical for clinical applications.
Strong task performance in clinical AI models can mask significant process instabilities, particularly in contradiction detection and hallucination management.
Existing medical AI benchmarks lack process visibility, atomic skill evaluation, and integrated hallucination detection. We introduce MedBench v5, a redesigned benchmark for clinical multimodal models (language, vision-language, and agent systems) that moves from static QA to dynamic, process-oriented evaluation. MedBench v5 features: (1) a dual-dimensional framework combining Clinical Cognitive Responsiveness (14 sub-dimensions) and Medical Atomic Skills (4 agent environments), covering 63 tasks; (2) three switchable information-flow stressors (omission, contradiction, evidence delay) for factorized degradation analysis; (3) a dynamic process audit protocol with five reasoning nodes that produces model-specific failure fingerprints; (4) hallucination propagation monitoring across initiation, propagation, anchoring, and contradiction interaction-capturing silent hallucination. Experiments on frontier models show that strong overall task performance does not guarantee process stability: stressors mainly disrupt contradiction detection, diagnosis updating, hallucination propagation, and contradiction-based self-correction, while final evidence grounding can remain superficially stable. MedBench v5 provides a unified infrastructure for capability profiling, controllable stress testing, process auditing, and hallucination trajectory analysis in clinical AI evaluation.
