Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces MIRTO, a novel evaluation protocol for unsupervised anomaly segmentation in brain MRI that explicitly gates comparisons with registration checks and label-free diagnostics, sets thresholds on validation data, and quantifies performance across 15,552 evaluation pipelines with bootstrap intervals. MIRTO reveals significant sensitivity of UAD method rankings to seemingly minor methodological choices, such as axis-order mismatches, and demonstrates that apparent metric advantages can be artifacts of threshold transfer. The protocol's application to four UAD methods on BraTS 2020 data highlights that while methods explain substantial variance in voxel-level metrics, lesion sensitivity is heavily influenced by lesion definition and hit criteria, and that training-free modifications can improve performance at equal false-positive burden.
Standard UAD evaluations for brain MRI are brittle, with a single axis-order mismatch capable of collapsing voxel AUROC from 0.873 to 0.583, while slice-level AUROC remains largely unaffected.
Unsupervised anomaly detection (UAD) methods for brain MRI are ranked by a single score, yet that score rests on choices that are rarely reported: how each anomaly map is aligned with the reference, how and on which data the threshold is set, and which false-positive budget, metric, aggregation and lesion definition are used. We present MIRTO, an evaluation protocol that makes these choices explicit and measures their effect. It gates the geometry of every comparison with a registration check and label-free diagnostics of known power, sets thresholds on validation data alone and reports the false-positive volume actually realised on test, repeats each comparison over 15,552 defensible evaluation pipelines, and attaches paired subject-bootstrap intervals with multiplicity control. Applied to four UAD methods trained on the same healthy data and tested on 312 BraTS 2020 subjects, MIRTO showed that an axis-order mismatch between stored maps and the reference lowered a diffusion model's voxel AUROC from 0.873 to 0.583 whilst barely moving its slice-level AUROC. Within each metric, the method explained at least 0.95 of the variance in voxel AUROC and AUPRC and 0.77 in Dice, but only 0.14 in lesion sensitivity, where the lesion definition and hit criterion dominated. A Dice advantage that was significant at validation thresholds vanished at equal realised false-positive burden, and an exact identity attributes it to threshold transfer. A training-free change to REFLECT's latent aggregation raised Dice at equal burden by 0.052. Nine hypotheses were tested against explicit criteria; because the same cohort served to develop the protocol, all inference is exploratory.