Search papers, labs, and topics across Lattice.
This paper investigates the reliability of uncertainty-based selective prediction in multimodal ICU data for multilabel clinical condition classification. The study reveals that selective prediction can significantly degrade performance despite strong standard evaluation metrics due to severe class-dependent miscalibration, where models are overconfident on incorrect predictions and underconfident on correct ones, especially for underrepresented conditions. The authors demonstrate that aggregate metrics can mask these issues, hindering proper assessment of selective prediction behavior in clinical AI.
Selective prediction, a proposed safeguard for AI in clinical settings, can backfire dramatically due to class-dependent miscalibration, leading to worse performance than simply trusting the model.
As artificial intelligence systems move toward clinical deployment, ensuring reliable prediction behavior is fundamental for safety-critical decision-making tasks. One proposed safeguard is selective prediction, where models can defer uncertain predictions to human experts for review. In this work, we empirically evaluate the reliability of uncertainty-based selective prediction in multilabel clinical condition classification using multimodal ICU data. Across a range of state-of-the-art unimodal and multimodal models, we find that selective prediction can substantially degrade performance despite strong standard evaluation metrics. This failure is driven by severe class-dependent miscalibration, whereby models assign high uncertainty to correct predictions and low uncertainty to incorrect ones, particularly for underrepresented clinical conditions. Our results show that commonly used aggregate metrics can obscure these effects, limiting their ability to assess selective prediction behavior in this setting. Taken together, our findings characterize a task-specific failure mode of selective prediction in multimodal clinical condition classification and highlight the need for calibration-aware evaluation to provide strong guarantees of safety and robustness in clinical AI.
