Search papers, labs, and topics across Lattice.
This study introduces a novel LLM-based Visual Explanation Evaluation Framework specifically designed to assess the quality of Grad-CAM explanations in facial skin disease classification models. By employing various augmentation strategies on models like EfficientNet-B0 and MobileNetV3, the research systematically evaluates whether the generated explanations accurately correspond to clinically relevant lesion regions. The key finding reveals that the LLM-as-a-Judge framework, utilizing advanced models such as GPT-5.5 and Claude Sonnet 4.6, enhances the evaluation of explanation trustworthiness and localization, thereby addressing a critical gap in explainability for medical AI applications.
LLMs can effectively evaluate the clinical relevance of AI-generated visual explanations in skin disease diagnosis, revealing critical insights into model trustworthiness.
This study proposes a domain-specific LLM-based Visual Explanation Evaluation Framework for assessing Grad-CAM explanations in facial skin disease diagnosis models. While previous studies have primarily focused on improving classification performance through data augmentation techniques, relatively few studies have systematically examined whether model explanations are grounded in clinically relevant lesion regions. In this study, geometric augmentation, color-based augmentation, and mixed augmentation strategies were applied to facial skin disease classification models based on EfficientNet-B0, MobileNetV3, and ResNet18. Grad-CAM was employed to generate visual explanations representing the models' decision-making processes. Furthermore, an LLM-as-a-Judge evaluation framework was designed using GPT-5.5, Gemini 3.5 Flash, and Claude Sonnet 4.6 to assess Grad-CAM explanations from the perspectives of lesion localization and explanation trustworthiness. To improve evaluation consistency and clinical grounding, a progressive prompt engineering strategy was introduced, incorporating evaluation rubrics, clinical knowledge, penalty rules, and structured output formats.
