Search papers, labs, and topics across Lattice.
The authors introduce Dual-IFM, an interpretable foundation model for retinal fundus images, designed with both local and global interpretability in mind. Local interpretability is achieved through class evidence maps, while global interpretability is enabled by a 2D projection layer for visualizing the model's representation space. Trained on a large dataset of fundus images, Dual-IFM achieves performance comparable to larger, less interpretable models, demonstrating the potential of combining SSL pretraining with inherent interpretability for robust representations in medical imaging.
You can get state-of-the-art performance on retinal fundus image tasks with an interpretable foundation model that's 16x smaller than the alternatives.
Foundation models are used to extract transferable representations from large amounts of unlabeled data, typically via self-supervised learning (SSL). However, many of these models rely on architectures that offer limited interpretability, which is a critical issue in high-stakes domains such as medical imaging. We propose Dual-IFM, a foundation model that is interpretable-by-design in two ways: First, it provides local interpretability for individual images through class evidence maps that are faithful to the decision-making process. Second, it provides global interpretability for entire datasets through a 2D projection layer that allows for direct visualization of the model's representation space. We trained our model on over 800,000 color fundus photography from various sources to learn generalizable, interpretable representations for different downstream tasks. Our results show that our model reaches a performance range similar to that of state-of-the-art foundation models with up to $16\times$ the number of parameters, while providing interpretable predictions on out-of-distribution data. Our results suggest that large-scale SSL pretraining paired with inherent interpretability can lead to robust representations for retinal imaging.
