Search papers, labs, and topics across Lattice.
This study explores the compression of large 3D foundation models, specifically MASt3R, through knowledge distillation for efficient lunar stereo reconstruction, addressing the critical limitations posed by hardware constraints in planetary exploration. By distilling the model into lightweight students with varying architectures, the authors introduce a structured SVD-based initialization to bridge the dimensional gap between teacher and student, leading to improved convergence and performance. The distilled models achieve up to 7 times reduction in size while retaining most of the teacher's accuracy, offering significant insights into the trade-offs between encoder and decoder configurations in geometric model distillation.
Distilling a 688M-parameter model into a lightweight version that retains 93% of its accuracy while reducing size by 7x could revolutionize 3D reconstruction in resource-constrained environments.
Large 3D foundation models such as MASt3R achieve state-of-the-art stereo reconstruction but are computationally demanding for deployment under strict hardware constraints -- a critical limitation in domains such as planetary exploration, where onboard computing is severely restricted. We study how far such models can be compressed through knowledge distillation, using lunar stereo reconstruction as a challenging and practically relevant case study. Starting from a 688M-parameter MASt3R teacher fine-tuned on lunar imagery, we distill its dense geometric predictions into a family of lightweight students spanning different encoder types (CNN vs ViT), decoder widths and depths, and training strategies. To bridge the dimensional mismatch between teacher and student, we propose a structured SVD-based initialization that projects the teacher's decoder weights into the student's smaller latent space, yielding a warm start that significantly improves convergence and final performance. Based on our results on lunar data, we can obtain a distilled student that retains most of teacher's reconstruction accuracy while reducing the model size up to 7 times, and even outperforms a baseline trained directly with sparse ground-truth annotations. Beyond compression, our study highlights both principles and practical insights for distilling geometric foundation models: a convolutional encoder underperforms transformer-based alternatives (though pretraining availability remains a confounding factor), preserving encoder capacity is more critical than maintaining a large decoder, feature-level distillation consistently outperforms output-only supervision, and SVD-based initialization improves optimisation stability. These findings provide practical guidelines for deploying 3D reconstruction models in resource-constrained environments.
