Search papers, labs, and topics across Lattice.
This paper introduces UniverSat, a novel Vision Transformer architecture designed to overcome the limitations of traditional patch projectors in Earth Observation tasks by employing a Universal Patch Encoder. This encoder allows for the integration of diverse input modalities and resolutions, enabling a single model to be trained on multimodal datasets through self-supervision. The approach demonstrates significant improvements in classification and segmentation performance on established Earth Observation benchmarks, showcasing its robustness and versatility across different sensor types.
UniverSat achieves sensor-agnostic feature extraction, outperforming traditional models in Earth Observation tasks by leveraging a flexible Universal Patch Encoder.
Vision Transformers (ViT) dominate computer vision. However, their reliance on rigid patch projectors hinders transfer to Earth Observation (EO), where input modalities, scales, and resolutions vary widely. We introduce UniverSat, a ViT-style backbone built around a Universal Patch Encoder that maps patches from arbitrary spatial, spectral, and temporal resolutions, and from both optical and non-optical sensors, into a shared embedding space with a shared set of weights. This enables training a single model on heterogeneous multimodal corpora via self-supervision, yielding robust, sensor-agnostic spatial features. We validate this approach with strong results across classification and segmentation on standard EO benchmarks from GeoBench, PANGEABench, and SpectralEarth. Our code and models are available at https://github.com/gastruc/UniverSat.
