Search papers, labs, and topics across Lattice.
This paper introduces the Boot-and-Feedback (BooF) framework, which enhances breast ultrasound diagnosis by facilitating collaboration between Multimodal Large Language Models (MLLMs) and expert models. By employing a structured Boot Stage that utilizes the BI-RADS lexicon and expert predictions, BooF effectively reduces hallucinations in MLLM outputs, while the Feedback Stage employs a lightweight Attention-Gated Cross-Modality Fusion Module to refine these outputs with visual features. Experimental results show that BooF significantly improves diagnostic accuracy and interpretability compared to existing state-of-the-art methods in breast ultrasound analysis.
MLLMs can be effectively guided to enhance diagnostic accuracy in breast ultrasound by leveraging expert feedback, reducing hallucinations and improving interpretability.
Breast ultrasound (BUS) is widely used for breast cancer diagnosis yet remains operator-dependent. While deep learning shows promise, ensuring diagnostic reliability and interpretability is challenging. Recent Multimodal Large Language Models (MLLMs) often generate spurious descriptions due to limited domain knowledge, which mislead downstream expert models and compromise clinical validity. To address these challenges, we propose the Boot-and-Feedback (BooF) model collaboration framework for synergistic MLLM-expert interaction. Specifically, in the Boot Stage, the MLLM is guided by the BI-RADS lexicon and preliminary benign-malignant vision-expert predictions, enabling it to transfer general reasoning to BUS analysis while avoiding hallucinations. Subsequently, the Feedback Stage integrates these descriptions with visual features via a lightweight Attention-Gated Cross-Modality Fusion Module. This allows the expert to leverage textual feedback while adaptively filtering noise. Extensive experiments on multiple BUS datasets demonstrate that BooF substantially outperforms state-of-the-art methods in terms of diagnostic accuracy and interpretability.
