Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper investigates "semantic drift" in chest X-ray classification models, where fine-tuning alters the visual evidence used for predictions despite maintaining accuracy. They quantified this drift by analyzing attribution maps (LayerCAM, GradCAM++) from DenseNet201, ResNet50V2, and InceptionV3 models fine-tuned on a five-class chest X-ray task, using metrics that capture spatial localization and structural consistency. Results show that while coarse anatomical localization remains stable, significant architecture-dependent reorganization of evidential structure occurs during fine-tuning, and stability rankings can even reverse across different attribution methods.
Fine-tuning can fundamentally rewire what a model "sees" in medical images, even when diagnostic accuracy holds steady, and this rewiring is heavily influenced by the model architecture itself.
Transfer learning followed by fine-tuning is widely adopted in medical image classification due to consistent gains in diagnostic performance. However, in multi-class settings with overlapping visual features, improvements in accuracy do not guarantee stability of the visual evidence used to support predictions. We define semantic drift as systematic changes in the attribution structure supporting a model's predictions between transfer learning and full fine-tuning, reflecting potential shifts in underlying visual reasoning despite stable classification performance. Using a five-class chest X-ray task, we evaluate DenseNet201, ResNet50V2, and InceptionV3 under a two-stage training protocol and quantify drift with reference-free metrics capturing spatial localization and structural consistency of attribution maps. Across architectures, coarse anatomical localization remains stable, while overlap IoU reveals pronounced architecture-dependent reorganization of evidential structure. Beyond single-method analysis, stability rankings can reverse across LayerCAM and GradCAM++ under converged predictive performance, establishing explanation stability as an interaction between architecture, optimization phase, and attribution objective.