Search papers, labs, and topics across Lattice.
This paper introduces Curia-MAE, a multi-modal, multi-anatomy masked autoencoder (MAE) pre-trained on a diverse dataset of 300,000 CT and MRI images to enhance 3D medical image segmentation. By incorporating a robust reconstruction objective, a feature regularizer, and a local-global similarity objective, Curia-MAE significantly outperforms existing frozen-encoder models, including the strong MAE baseline, particularly in scenarios with limited labeled data. The findings suggest that a single frozen encoder can effectively generalize across various segmentation tasks, streamlining the adaptation and deployment of models in clinical settings.
Curia-MAE achieves superior performance in 3D medical image segmentation with a frozen encoder, challenging the need for extensive fine-tuning even in data-scarce environments.
Radiology foundation models learn transferable representations that can be adapted to new tasks by training only small layers on top of a frozen encoder. Dense prediction tasks such as 3D segmentation are, however, underrepresented in their evaluation, and, with the encoder kept frozen, pre-trained models still fall short of nnU-Net, the state-of-the-art reference trained from scratch. To close this gap we extend convolutional MAE pre-training with a robust reconstruction objective, a feature regularizer, and a local-global similarity objective. Using this method, we propose Curia-MAE, a multi-modal, multi-anatomy MAE model pre-trained on 300,000 CT and MRI images covering a large number of anatomical sites. On eight anatomy- and lesion-focused segmentation benchmarks, Curia-MAE improves frozen-encoder performance over a strong MAE baseline, while remaining competitive under full finetuning and superior on lesion tasks, where labeled data is scarce. These results indicate that a single frozen encoder can be reused across diverse segmentation tasks, reducing the cost of adapting and deploying such models in clinical workflows. We will make our pre-trained model weights publicly available.
