Search papers, labs, and topics across Lattice.
This paper investigates the zero-shot diagnostic capabilities of multimodal large language model (MLLM) agents in distinguishing visually similar diseases, specifically melanoma vs. atypical nevus and pulmonary edema vs. pneumonia. They introduce a multi-agent framework using contrastive adjudication to improve diagnostic accuracy and reduce unsupported claims. Experiments demonstrate an 11% accuracy improvement on dermoscopy data, suggesting potential for MLLM agents in challenging diagnostic tasks, though performance is not yet clinically viable.
Even with multi-agent adjudication, MLLMs still struggle to reliably differentiate visually confounded diseases in a zero-shot setting, highlighting limitations in current agent-based diagnostic approaches.
The rapid progress of multimodal large language models (MLLMs) has led to increasing interest in agent-based systems. While most prior work in medical imaging concentrates on automating routine clinical workflows, we study an underexplored yet clinically significant setting: distinguishing visually hard-to-separate diseases in a zero-shot setting. We benchmark representative agents on two imaging-only proxy diagnostic tasks, (1) melanoma vs. atypical nevus and (2) pulmonary edema vs. pneumonia, where visual features are highly confounded despite substantial differences in clinical management. We introduce a multi-agent framework based on contrastive adjudication. Experimental results show improved diagnostic performance (an 11-percentage-point gain in accuracy on dermoscopy data) and reduced unsupported claims on qualitative samples, although overall performance remains insufficient for clinical deployment. We acknowledge the inherent uncertainty in human annotations and the absence of clinical context, which further limit the translation to real-world settings. Within this controlled setting, this pilot study provides preliminary insights into zero-shot agent performance in visually confounded scenarios.
