Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study addresses the shortcomings of open-vocabulary segmentation in remote sensing imagery by identifying that the text queries used for segmentation often fail to effectively map to the vision-language embedding space. By employing textual inversion on a frozen model, the authors enhance the quality of the text queries, leading to a significant improvement in segmentation performance. The mean intersection over union for affected categories increased from 3.9 to 39.4, outperforming existing few-shot methods that utilize visual prompts at inference.
Textual inversion can transform weak text queries into powerful segmentation tools, boosting performance in remote sensing tasks dramatically.
Open-vocabulary segmentation labels arbitrary categories from a text query without per-class training, yet on remote sensing imagery it underperforms on categories it handles reliably elsewhere. We find that much of this gap traces to the text query rather than to the segmentation model. Because these models are not specialized for overhead imagery, the class name that serves as the query is often a weak address into the vision-language embedding space. We show that a better name repairs part of the gap, while the remaining failures call for an address that the tested natural-language rephrasings do not provide. We recover that address from a few examples through textual inversion on a frozen model, keeping inference text only. On a representative benchmark this raises the mean intersection over union on the affected categories from 3.9 to 39.4, and across eight remote sensing datasets it improves over few-shot methods that instead inject visual prompts at inference.