Search papers, labs, and topics across Lattice.
This study introduces PanDent, a comprehensive benchmark for evaluating multimodal large language models (MLLMs) in dental panoramic radiography, featuring 9,524 expert-annotated OPGs. The dataset facilitates the assessment of whether MLLMs can generate clinically consistent radiology reports that align with expert interpretations at the tooth level. Results indicate that while MLLMs can produce fluent reports, they often lack clinical consistency, but fine-tuning on PanDent significantly enhances their diagnostic accuracy and localization capabilities.
Current MLLMs generate fluent dental reports but often miss the mark on clinical consistency, revealing a critical gap in AI's diagnostic capabilities.
Accurate evaluation of multimodal large language models (MLLMs) in dental panoramic radiography (orthopantomogram, OPG) is limited by the lack of fine-grained, clinically reliable benchmarks that reflect expert interpretation. This work introduces PanDent, a large-scale, clinically grounded OPG benchmark built upon fine-grained, expert-validated tooth-level annotations. The dataset comprises 9,524 high-quality OPGs, each associated with comprehensive structured annotations produced by experienced dentists and further validated by an oral and maxillofacial radiologist, providing clinically reliable supervision for tooth-level diagnosis and reasoning. Clinically consistent radiology reports are constructed from expert-validated findings using clinician-defined reporting logic, establishing explicit correspondence between structured clinical evidence and free-text descriptions. This design enables evaluation of whether MLLMs generate reports that are not only linguistically coherent but also clinically consistent with expert-validated tooth-level findings. Experiments are conducted on diverse MLLMs, including state-of-the-art (SOTA) proprietary models, general-domain open-source models, and medical-specific models. Results show that current MLLMs can generate fluent reports, yet fail to produce clinically consistent descriptions, exhibiting substantial errors in fine-grained localization and tooth-level diagnosis. Fine-tuning on PanDent significantly improves structure-language consistency, substantially enhancing visual localization accuracy and diagnostic correctness, and bringing model outputs closer to expert dental interpretation. These results establish PanDent as a rigorous benchmark for evaluating tooth-level clinical reasoning in MLLMs and a valuable resource for clinically grounded dental AI.
