Search papers, labs, and topics across Lattice.
This paper introduces MetaStructAtlas, a comprehensive dataset designed for the joint interpretation of metabolic function and anatomical structure in whole-body PET/CT imaging, addressing a significant gap in existing research. The dataset includes 490 co-registered 3D PET and CT volumes, complete with 50,470 organ-level segmentation masks and grounded radiology reports, enabling detailed multimodal analysis. Additionally, the authors present MetaStructVQA, a benchmark for visual question-answering that links diagnostic queries to visual evidence, thereby enhancing the evaluation of state-of-the-art 3D medical vision-language models in nuclear medicine.
A groundbreaking dataset and benchmark that bridges the gap between anatomical and metabolic analysis in whole-body PET/CT imaging, enabling advanced multimodal reasoning.
The joint interpretation of metabolic function and anatomical structure is essential for clinical diagnosis in whole-body PET/CT. Although recent advances in 3D medical vision-language models have demonstrated remarkable progress, current efforts are limited to regional CT imaging, leaving a critical void in comprehensive whole-body PET/CT analysis. In this work, we introduce MetaStructAtlas, a large-scale dataset for grounded whole-body PET/CT interpretation that synthesizes multimodal imaging with integrated anatomical, metabolic, and semantic annotations. MetaStructAtlas provides 490 co-registered 3D PET and CT volumes with 50,470 organ-level segmentation masks and grounded radiology reports. To facilitate interactive reasoning, we further developed MetaStructVQA, a standardized 3D grounded visual question-answering benchmark containing 100,565 QA pairs. This framework explicitly links diagnostic queries to visual evidence across modalities, encompassing anatomical, morphological, and metabolic characteristics. Finally, we evaluate state-of-the-art 3D medical VLMs on MetaStructVQA, establishing a robust foundation for multimodal representation learning and integrated whole-body reasoning in nuclear medicine.
