Search papers, labs, and topics across Lattice.
This paper introduces a comparative explainability framework to audit DeBERTa-v3's zero-shot medical abstract classification, addressing the disagreement problem in XAI by comparing five attribution methods (SHAP, LIME, occlusion, Input x Gradient, Attention x Gradient). The study quantizes pairwise explanation agreement using the Jaccard index on a corpus enriched with hypotheses, revealing that explanatory stability strongly correlates with predictive certainty, particularly degrading under semantic ambiguity. Qualitative analysis identified lexical hypersensitivity, semantic overlap, and loss of attribution coherence as key failure modes, advocating for multi-method explanation auditing and ontology prioritization.
Transformer model explanations diverge significantly under diagnostic uncertainty, highlighting the critical need for multi-method auditing and careful ontology selection in high-stakes medical classification.
A comparative explainability framework is presented to audit DeBERTa-v3 under zero-shot classification of medical abstracts. The work addresses the disagreement problem in Explainable Artificial Intelligence, where different attribution methods produce divergent explanations for the same input and prediction. A natural language inference engine is implemented over the Medical Abstracts corpus with five enriched hypotheses per diagnostic category and a balanced sample of one thousand texts per class. Five explanation methods are compared: SHAP and LIME as model-agnostic approaches, occlusion and Input x Gradient as deep-learning-specific approaches, and Attention x Gradient as a transformer-specific approach. Explanations are standardized through top-token attribution, and pairwise agreement is quantified using the Jaccard index. High predictive accuracy is achieved across well-defined clinical domains, whereas performance degrades under high semantic ambiguity. Explanatory stability directly mirrors predictive certainty, exhibiting strong convergence in univalent categories and a marked drop under diagnostic uncertainty. Furthermore, qualitative error auditing uncovers three systemic failure mechanisms: lexical hypersensitivity, semantic overlap, and loss of attribution coherence. The results support the combined use of several explanation methods and quantitative agreement metrics when auditing transformer-based models in medical text classification, and suggest prioritizing specific clinical ontologies over broad diagnostic labels.
