Search papers, labs, and topics across Lattice.
This study evaluates the effectiveness of conditional diffusion models for generating synthetic histopathology images, addressing the inadequacies of traditional evaluation metrics like FID and IS when applied to medical data. By employing modified versions of these metrics, which leverage foundation models trained on digital pathology datasets, the authors demonstrate that pathology-specific metrics correlate better with downstream task performance, particularly in nuclei segmentation. The findings reveal that enhancing the diversity of synthetic training data significantly improves segmentation model performance, overshadowing the importance of visual fidelity in individual images.
Pathology-specific evaluation metrics reveal that increasing synthetic data diversity boosts segmentation model performance more than visual fidelity does.
Synthetic histopathology image generation has emerged as an approach that may address data scarcity in computational pathology, yet current evaluation methodologies may not fully assess synthetic data quality for medical applications. This work investigates and addresses limitations in existing evaluation metrics, investigating an approach for assessing synthetic histopathology image quality through domain-specific metrics and downstream task validation. We show that conventional synthetic data evaluation metrics such as Frechet Inception Distance (FID) and Inception Score (IS) may have limitations when applied to histopathology images due to their reliance on ImageNet-pretrained feature extractors. To address these limitations, we propose for consideration modified FID and IS approaches utilizing foundation models pretrained on digital pathology datasets, supplemented by precision-recall based metrics as part of an additional quality assessment. Using conditional denoising diffusion models trained on four benchmark datasets, with a two-step training approach, we generated synthetic datasets with systematically varied quality characteristics. We also measured the correlation between the synthetic data quality metrics with downstream nuclei segmentation performance using common metrics including the aggregated Jaccard index (AJI+) and the Dice coefficient. The study results suggest that pathology-specific metrics may provide improved discriminative power. Specifically, the modified Inception Score indicates higher correlation with downstream task performance (r=0.6096 with AJI+, p=0.0122), compared to the original IS (r=0.0708, p=0.7944). Our observations indicate that increasing the variety of generated training data has a higher positive correlation with segmentation model performance than improving the visual fidelity of individual generated images.
