Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces DualIFM, an interpretable foundation model designed specifically for retinal fundus images, utilizing a BagNet backbone to generate class evidence maps that clarify the model's decision-making. By incorporating a 2D projection layer during pretraining, DualIFM allows for direct visualization of the representation space, revealing clinically meaningful clusters and potential spurious correlations. Trained on over 800,000 color fundus photographs, DualIFM achieves performance on par with much larger models while maintaining interpretability, highlighting the potential for robust, explainable representations in medical imaging.
Achieving comparable performance to a 16x larger model, DualIFM offers interpretable predictions that illuminate the decision-making process in retinal imaging.
Foundation models are used to extract transferable representations from large amounts of unlabeled data, typically via self-supervised learning (SSL). However, many of these models rely on architectures that offer limited interpretability, a critical issue in high-stakes domains such as medical imaging. We propose DualIFM, a foundation model that is interpretable-by-design via a BagNet backbone whose small receptive fields generate class evidence maps that are faithful to the model's decision-making process. Additionally, DualIFM incorporates a 2D projection layer during pretraining that enables direct visualization of the representation space, providing a dataset-level view of the learned structure including meaningful clinical clusters as well as potential spurious correlations. We trained DualIFM on over 800,000 color fundus photographs from various sources to learn generalizable representations for different downstream tasks. Our model achieves performance comparable to RETFound, which has 16times more parameters, while providing interpretable predictions on out-of-distribution data. These results suggest that large-scale SSL pretraining paired with inherent interpretability can lead to robust representations for retinal imaging. Code and pretrained models are available at github.com/berenslab/interpretable_FM.