Search papers, labs, and topics across Lattice.
This study introduces a lightweight adaptation method for general-purpose vision-language models (VLMs) to effectively handle multispectral and synthetic aperture radar (SAR) images by rendering each observation as multiple optical views alongside a SAR view. By leveraging a multi-image interface and applying Low-Rank Adaptation (LoRA) to specific visual transformer blocks, the authors achieve significant performance improvements on land-cover recognition tasks without the need for extensive retraining of the foundational models. The adapted Qwen3-VL model achieves a micro F1 score of 0.8275 on a balanced benchmark, showcasing the method's efficacy across various VLM architectures and applications, including flood verification and image captioning.
VLMs can be repurposed for multispectral and SAR tasks with minimal adaptation, achieving impressive accuracy without retraining from scratch.
General-purpose vision-language models (VLMs) now support strong visual recognition, instruction following, and generation. However, most pretrained visual encoders are built around three-channel natural images and do not directly accommodate observations such as native multispectral measurements or synthetic aperture radar (SAR). Adapting VLMs to these sensors typically requires dedicated encoders and domain pretraining, slowing the reuse of stronger general-purpose checkpoints. We show that the multi-image interface of general-purpose VLMs offers a lightweight alternative. Our protocol renders each observation as five optical views and one SAR view, names them in the prompt, and adapts the language network and selected visual transformer blocks with LoRA. This exposes band composites, spectral indices, and radar backscatter through an existing visual interface. For land-cover recognition, structured supervision couples predicted classes with sensor evidence. We further construct preference pairs in which a true label is omitted while its supporting evidence is retained, encouraging complete predictions that remain consistent with the observations. On a balanced six-class land-cover benchmark derived from BigEarthNet-v2, the adapted Qwen3-VL reaches 0.8275 micro F1. The same input and adaptation protocol improves all four tested VLM architectures and transfers to Sen1Floods11 flood verification and BigEarthNet.txt captioning. Image removal and mismatch controls show that the adapted models use the supplied sensor observations. Together, these results demonstrate that VLMs can be repurposed for multispectral and SAR tasks through rendered inputs and compact LoRA adaptation, without training a new foundation model.
