Search papers, labs, and topics across Lattice.
This paper introduces LocAnyMed-200K, a comprehensive multimodal medical visual grounding dataset with approximately 200K image-query-answer pairs across various imaging modalities, addressing the fragmentation in existing medical localization resources. By fine-tuning the LocateAnything-3B model on this dataset, the authors achieve a significant improvement in localization performance, with F1@IoU 0.50 rising from 10.64 to 85.59 on a held-out evaluation set. Additionally, they present LocAnyMed-CoT-20K, a rationale-augmented subset that enhances cross-source generalization by linking anatomical context and visual observations through structured reasoning.
Fine-tuning a general grounding model on a newly constructed dataset boosts medical localization accuracy from 10.64 to 85.59, showcasing the power of domain-specific supervision.
Medical visual grounding connects free-form clinical queries to spatial evidence in medical images and is an important component of interpretable medical artificial intelligence. However, general-purpose grounding models are predominantly trained on natural images, while existing medical localization resources remain fragmented across imaging modalities, datasets, and task formulations. To address this gap, we construct LocAnyMed-200K, a multimodal medical visual grounding dataset containing approximately 200K image-query-answer examples across computed tomography, optical medical imaging, ultrasound, and X-ray. We harmonize heterogeneous detection and localization resources into a unified free-form instruction format that supports one or multiple bounding boxes, point coordinates, and no-target outputs for negative queries. Full-parameter fine-tuning of LocateAnything-3B on LocAnyMed-200K improves F1@IoU 0.50 from 10.64 to 85.59 on a held-out evaluation split, demonstrating that large-scale domain-specific supervision can equip a general grounding model with effective medical localization capabilities. Beyond spatial coordinates, a clinically interpretable grounding system should also communicate the evidence supporting its prediction. We therefore derive LocAnyMed-CoT-20K, a rationale-augmented subset that connects anatomical context, visual observations, and spatial conclusions through structured reasoning and further improves cross-source generalization through fine-tuning. Together, these resources provide a unified foundation for studying both localization accuracy and rationale quality across heterogeneous medical imaging modalities. The code is publicly available at https://github.com/MiliLab/LocAnyMed.
