Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study introduces a two-branch multimodal network that integrates raw MRI scans with 91 radiomic features to classify brain tumors into four categories: glioma, meningioma, pituitary, and no-tumor. By employing a pre-trained CNN for image encoding and a dedicated MLP for radiomic feature encoding, the model effectively mimics the multimodal reasoning of clinicians. The results demonstrate that all multimodal configurations surpass unimodal baselines, with the gated fusion method achieving the highest accuracy of 96.13% across a balanced dataset of 7,200 images.
Gated fusion of imaging and radiomic features leads to a remarkable 96.13% accuracy in brain tumor classification, outperforming traditional unimodal approaches.
Clinicians diagnose brain tumors by synthesizing patient symptoms, medical history, and quantitative imaging data from modalities such as MRI and CT scans into a unified clinical judgement. However, most deep learning models rely on MRI/CT images alone, failing to replicate the clinicians multimodal reasoning. We explore a two-branch multimodal network combining raw MRI scans with 91 extracted radiomic features (intensity, texture, shape, and boundary descriptors) to classify brain tumors into glioma, meningioma, pituitary, and no-tumor. A pre-trained CNN backbone encodes the image stream, whereas a dedicated MLP encodes the radiomic stream. Both streams are fused via concatenation, gated, or bidirectional cross-modal attention strategies. Across nine experimental runs on a balanced 7,200 image dataset, all multimodal configurations outperform unimodal baselines with gated fusion achieving the best accuracy of 96.13%.