Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
MR-JEPA is a self-supervised video foundation model specifically designed for cardiac magnetic resonance imaging (CMR), addressing the limitations of existing models that only process individual 2D slices. By utilizing tubelet tokenization and spatiotemporal masking, MR-JEPA is pretrained on a diverse dataset of multi-sequence CMR data from over 10,000 patients, significantly enhancing its performance across various cardiac assessment tasks. The model outperforms existing methods in both regression tasks and disease detection, demonstrating its effectiveness in leveraging the rich contextual information inherent in CMR data for clinical applications.
MR-JEPA achieves a remarkable 21-27% reduction in mean absolute error on key cardiac strain tasks compared to existing models, showcasing the power of unified video encoding in medical imaging.
Cardiac magnetic resonance imaging (CMR) produces rich sequential data such as temporal cine videos and spatial LGE/mapping stacks, yet most deep learning approaches process individual 2D slices, discarding this context. We present MR-JEPA, a self-supervised video foundation model for CMR that extends LeJEPA to 3D spatiotemporal inputs through tubelet tokenization, spatiotemporal masking augmentation, and initialization from a 2D CMR foundation model. Unlike prior CMR video models limited to cine data, MR-JEPA is pretrained on multi-sequence data (cine, LGE, mapping) from 10,505 patients across two centers without annotations. We evaluate the frozen encoder on six downstream tasks using a unified multi-view gated attention architecture: LV ejection fraction, RV ejection fraction, three myocardial strains (GLS, GCS, GRS), and four-class disease detection. MR-JEPA outperforms other compared methods on all five regression tasks, including both a domain-specific CMR model pretrained on more data with text supervision and a natural-video foundation model, achieving an LV EF MAE of 4.79% (r =0.764) and a GLS MAE of 1.87 (r=0.805), with 21-27% MAE reductions over baselines on strain tasks. For disease detection, MR-JEPA achieved a macro AUG of 0.868, remaining competitive with the domain-specific baseline despite using a fully self-supervised pretraining objective. These results demonstrate the potential of a unified video encoder for robust, multi-view utilization of diverse CMR sequences in clinical cardiac quantification and diagnosis.
TU Munich