Search papers, labs, and topics across Lattice.
This study introduces ConfTriage, a calibration-aware framework utilizing a generalist large language model (LLM) to triage pulmonary nodule malignancy based solely on natural-language descriptions of nodule attributes. The framework effectively combines LLM inference with a selective referral to specialist deep learning models for low-confidence cases, achieving an F1 score of 88.22% and an AUC of 0.92. Notably, the research reveals that natural-language descriptions provide a more significant diagnostic signal than low-level image statistics, highlighting the potential of LLMs in medical decision-support systems.
Natural-language descriptions can outperform traditional image statistics in predicting pulmonary nodule malignancy, enabling a calibrated LLM to triage cases effectively.
Pulmonary nodule malignancy prediction typically depends on image-trained specialist deep learning (DL) models that require substantial annotated imaging data and task-specific training. We investigate whether a generalist large language model (LLM), reading only a faithful natural-language rendering of standard nodule attributes, can serve as a calibrated triage layer. We propose ConfTriage, a confidence-calibrated method built on three pillars: language as the modality, calibration as the safety mechanism, and a selective specialist DL backstop for low-confidence cases. We prove two guarantees: a finite-sample combined-error bound yielding an explicit per-threshold operational certificate, and an oracle inequality showing that excess risk over the Bayes-optimal deferral classifier is controlled by the L1 calibration error of the LLM probability. A controlled seven-way input ablation across five frontier LLMs on LIDC-IDRI shows that natural-language descriptions dominate the diagnostic signal, while low-level image statistics are essentially diagnostically vacuous. ConfTriage achieved an F1 score of 88.22% and an AUC of 0.92, resolving 76.5% of cases using zero-shot LLM inference alone and referring only uncertain cases to the specialist DL backstop. These results demonstrate that clinically meaningful diagnostic information can be captured through structured radiological descriptions and leveraged by calibrated LLMs for selective referral. The framework suggests a practical pathway for combining generalist LLM prediction with specialist AI models in medical decision-support systems. Source code is publicly available at https://github.com/rabiul-ai/ConfTriage.
