Search papers, labs, and topics across Lattice.
This study analyzes the impact of three parameter-efficient adaptation strategies鈥攍inear heads, MLPs, and attention pooling鈥攐n the performance and subgroup fairness of a frozen Rad-DINO chest X-ray encoder using the MIMIC-CXR dataset. The findings reveal that while attention pooling yields the best overall classification performance, it does not consistently mitigate subgroup disparities across race, sex, and imaging-view subgroups. Notably, stronger encoding of protected attributes like race does not correlate with reduced performance gaps, highlighting the complex relationship between model performance and fairness in medical imaging tasks.
Attention pooling enhances performance but paradoxically amplifies subgroup disparities, challenging assumptions about fairness in model adaptation.
Foundation models are increasingly adapted for downstream medical imaging tasks, yet the influence of the chosen adaptation strategy on subgroup fairness remains poorly understood. We investigate how three parameter-efficient adaptation techniques, including linear heads on the raw CLS token, an MLP, and an attention-pooling module over multi-layer patch features, affect both pathology classification performance and subgroup disparities when applied to the frozen Rad-DINO chest X-ray encoder. Using MIMIC-CXR, we evaluate eight pathologies across race, sex, and imaging-view subgroups on a prevalence-preserving, demographically balanced test set, and additionally probe how strongly each adapter encodes protected attributes. We find that attention pooling achieves the strongest overall discriminative performance and encodes attributes, particularly race, most strongly, but that improved overall performance does not consistently reduce subgroup disparities. Notably, stronger attribute encoding did not correspond to larger disparities: early network layers encoded race most weakly yet produced the largest subgroup performance gaps. Exploring different attention-pooling layer combinations further revealed no consistent relationship between the layers pooled, attribute encoding strength, and subgroup fairness. Our results indicate that richer, more expressive representations can improve accuracy while leaving fairness implications task-dependent and unpredictable, which must be assessed directly and per-task rather than inferred from encoding strength or overall performance alone.
