Search papers, labs, and topics across Lattice.
The study introduces TMF-RSE, a tri-modal deep learning framework that integrates appearance features from chest images, structural features from lung segmentation masks, and semantic features from vision-language models to accurately quantify lung disease severity. This approach is significant as it combines complementary fusion mechanisms and evidential regression to not only predict severity but also provide uncertainty estimates, enhancing clinical decision-making. Experimental results demonstrate TMF-RSE's superiority over recent transformer-based models, achieving impressive metrics on both the Per-COVID-19 CT and RALO datasets.
TMF-RSE achieves unprecedented accuracy in lung severity scoring while quantifying uncertainty, setting a new benchmark for clinical imaging applications.
Accurate quantification of lung disease severity from chest imaging is critical for clinical decision-making and resource allocation. We propose a tri-modal deep learning framework, TMF-RSE (Tri-Modal Fusion with Regional Semantics and Evidential Uncertainty), that combines appearance features from two-dimensional chest inputs, structural features from lung segmentation masks, and semantic features from vision-language models (VLMs) for severity quantification. Our approach employs complementary fusion mechanisms that integrate semantic guidance, structural priors, and hierarchical interactions across modalities. The model employs evidential regression to provide both severity predictions and uncertainty estimates. Experiments on the Per-COVID-19 CT and RALO datasets show that TMF-RSE outperforms recent transformer-based baselines, achieving MAE of 4.02 and Pearson correlation of 0.9629 on Per-COVID-19 validation, and 0.339 MAE / 0.973 PC on RALO geographic extent.
