Search papers, labs, and topics across Lattice.
This study evaluates the entire unsupervised domain adaptation (UDA) pipeline for medical imaging by analyzing eleven cross-domain scenarios across nine datasets, utilizing ten UDA algorithms and 13 label-free selection methods. The findings reveal that while capable adapted models exist, reliably identifying them without target labels is challenging, as validator-selected models exhibit a significant performance gap. The research suggests that improving the selection step through strategies like ensembling and limited target-labeling can help bridge this gap, highlighting the importance of a comprehensive approach for clinical deployment of UDA.
Identifying the right model for clinical deployment without target labels is fraught with challenges, leaving a significant performance gap even with advanced selection methods.
Deploying unsupervised domain adaptation (UDA) in clinical practice requires choosing which algorithm to use and which of its trained models to ship. However, the deployment (target) domain is unlabeled, so models cannot be evaluated directly on it, leaving it unclear which to select. We address this by evaluating the complete UDA pipeline, considering both adaptation and label-free selection together. Our study covers eleven clinically relevant cross-domain scenarios from nine medical imaging datasets, with ten UDA algorithms and 13 label-free selection methods (validators), evaluating over 80,000 trained models in total. By this, we find that a capable adapted model usually exists, but identifying it without target labels is difficult: the validator-selected models leave a large and structural target performance gap to the best available one, with no evaluated validator consistently reliable. Towards closing it, we explore two strategies, ensembling and a small target-labeling budget; both narrow this gap but do not close it entirely. Overall, deployable UDA depends on the complete pipeline; addressing the less explored selection step could bring much of current UDA closer to clinical use.
