Search papers, labs, and topics across Lattice.
This study conducts the first fairness audit of cervical-spine MRI segmentation, revealing that while the deployed model appears demographically fair, the reference labels used for evaluation introduce significant bias. The authors find that performance metrics can be inflated by approximately 8 Dice points when evaluated against machine-generated silver labels instead of expert-annotated gold labels. This highlights the critical issue of reference-label provenance, which can distort both performance assessments and fairness claims in segmentation tasks.
Relying on machine-generated labels can lead to inflated performance metrics and misleading fairness conclusions in medical imaging tasks.
Automated segmentation of cervical-spine MRI is increasingly used in clinical workflows, yet no fairness audit exists for this anatomy. We show that auditing these segmentation tasks is complicated by a common property of modern segmentation datasets: expert-annotated gold labels are expensive, so abundant machine-generated (silver) labels are added to limit annotation cost. This matters because the reference used to judge a model can itself be biased. In this study, we present the first fairness audit of cervical-spine MRI segmentation across sex, age, and race using the CSpineSeg dataset. We observe that the deployed model is demographically fair, but the choice of reference label, however, is not neutral. Because a dataset's silver labels are generated by a model trained on its gold labels, any new model trained on those same gold labels agrees more with the silver labels than with expert truth: scoring identical predictions against silver rather than gold overestimates performance by ~8 Dice points and turns the fairness verdict for age from non-significant to significant - not by the gap inflation Parikh et al. report (which we term false magnitude) but by collapsing within-group variance (which we term false confidence). Reference-label provenance is thus a first-order confounder in segmentation evaluation: performance and fairness should be reported against expert labels, and any fairness claim stated together with the provenance of its reference.
