Search papers, labs, and topics across Lattice.
This study rigorously evaluates the reliability of deep learning models for brain tumor segmentation by analyzing both a baseline nnU-Net and a deep ensemble approach under varying input conditions. The findings reveal that while the ensemble model shows improved calibration in in-distribution scenarios, it significantly outperforms the single model in detecting errors under synthetic acquisition shifts, highlighting the importance of inter-member disagreement as a robust indicator of model uncertainty. Ultimately, the research underscores the critical need for models to not only perform well but also to accurately convey their uncertainty in clinical settings, especially when faced with data that diverges from training distributions.
Inter-member disagreement in deep ensembles serves as a more sensitive indicator of model uncertainty than single-model confidence, especially under data shifts.
Deep networks segment brain tumours accurately in-distribution, but can fail silently when the input differs from their training data. That risk is central to clinical deployment and is the premise of the BraTS-GoAT generalizability task. We ask not only how well a model segments, but whether its uncertainty knows when it is wrong. On BraTS-GoAT (Task 3) we train a 5-fold cross-validated nnU-Net baseline (one held-out prediction per case) and a 3-seed deep ensemble. Both are evaluated for calibration and error detection on a per-region relevant mask, aggregated per case. In-distribution the 3-seed ensemble improves modestly over the already strong single model on the same held-out split, with the clearest gain in calibration. The separation appears under shift. In a controlled robustness study using graded synthetic corruptions as a proxy for acquisition shift, the single model's confidence stays flat while its accuracy and calibration degrade. Inter-member disagreement instead rises steeply, about a quarter to a third above the clean condition, several times the single model's response. On the official validation leaderboard the 5-fold ensemble of those folds attains whole-tumour Dice 0.87. The generalization gap is concentrated on the harder regions, with a characteristic failure of missing small, satellite lesions on unseen cohorts. In the synthetic study, disagreement among the 3-seed members is a more sensitive case-level indicator of acquisition shift than single-model confidence. Its per-voxel error localisation weakens as severity grows. The contribution is a rigorous, honest reliability comparison rather than a claim that any one uncertainty method dominates.
