Search papers, labs, and topics across Lattice.
This paper introduces a classifier-free method for generating Visual Counterfactual Explanations (VCEs) using Contrastive Analysis (CA), which disentangles common and salient generative factors across different datasets. By operating on data distributions rather than decision boundaries, the approach mitigates biases associated with specific classifiers, resulting in more robust counterfactual images. Evaluations on three medical imaging datasets reveal that this method significantly outperforms existing VCE techniques in terms of generation quality.
Counterfactual images generated without reliance on specific classifiers can significantly reduce bias and improve interpretability in medical imaging tasks.
Visual Counterfactual Explanations (VCEs) aim to explain image classifiers by generating minimally edited and realistic versions of an input image that change the classifier's prediction. Existing VCE methods are inherently classifier-dependent and therefore susceptible to classifier biases and failure modes, such as sensitivity to shortcut features and calibration errors. In this paper, we propose a classifier-free approach for visual counterfactual generation based on Contrastive Analysis (CA). Given two datasets corresponding to different classes (e.g., healthy and patients), we disentangle the generative factors that are common across the two datasets from those that are salient to each dataset, and generate counterfactual images by swapping only the salient factors. By operating directly on data distributions rather than decision boundaries, our method provides model-agnostic VCEs that are less sensitive to classifier biases. Our approach leverages the high-quality synthesis and well-structured latent space of StyleGAN2. We use the feature space F, instead than the usual W-space, to improve detail preservation. Unlike conventional CA approaches, which typically assume salient factors in only one dataset, we introduce an adapted framework and loss functions for VCE that allow multiple salient factors in each dataset. We evaluate our method on three medical imaging datasets and demonstrate superior counterfactual generation quality compared to existing approaches.
