Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
CoM$^3$eT is a novel medical vision foundation model that integrates multiple specialties and output types by leveraging attention mechanisms to model multidimensional context. This model demonstrated superior performance in an open competition across various datasets, achieving results comparable to full fine-tuning while training with less than 2.5% of the parameters. Additionally, CoM$^3$eT's federated learning capabilities allow it to maintain high performance using consumer-grade hardware, making it accessible for diverse clinical applications without the need for extensive computational resources.
Achieving top-tier medical image analysis performance with less than 2.5% of the parameters required for full fine-tuning could revolutionize accessibility in healthcare AI.
Medical foundation models improve generalization when training AI models with limited labeled data, but remain confined to a single specialty, such as pathology or radiology, and to either sparse or dense outputs, such as classification or segmentation. Here, we present CoM$^3$eT (Co-representation Multidimensional Multitask Medical Transformer), a medical vision foundation model that unifies pathology and radiology, sparse and dense predictions, and two- and higher-dimensional inputs by modeling multidimensional context with attention. CoM$^3$eT outperformed other medical foundation models in an open competition spanning five tomographic, four whole-specimen, and three two-dimensional datasets, covering sparse and dense prediction tasks as well as report generation. When adapted across diverse clinical applications, training fewer than 2.5% of parameters achieved performance comparable to full fine-tuning, enabling research without access to high-performance GPU clusters. Applied to federated learning across hospitals, this approach achieved performance comparable to pooled-data training over internet connections and with consumer-grade hardware.