Search papers, labs, and topics across Lattice.
This paper introduces UniRef-UAV, a multimodal benchmark designed to enhance visual grounding capabilities in UAV imagery by accommodating diverse query modalities and target cardinalities. The benchmark addresses limitations of existing referring expression comprehension methods, enabling the processing of text-only, image-only, and combined queries with varying target instances. The UAV-URNet detection-style baseline demonstrates improved performance in no-target discrimination and offers a more efficient implementation compared to large general-purpose models, highlighting the benefits of multimodal queries in reducing ambiguity and enhancing query-target alignment.
Multimodal queries in UAV imagery can significantly reduce visual-query ambiguity and improve target localization across diverse scenarios.
Unmanned aerial vehicles (UAVs) increasingly rely on visual grounding capabilities to localize task-relevant targets from diverse instructions in complex aerial scenes. Existing referring expression comprehension (REC) benchmarks and methods, however, are largely built around text-only queries and single-object outputs, which limits their applicability to practical UAV scenarios involving reference images, multimodal instructions, absent targets, and multiple valid target instances. To address this gap, we introduce \emph{Universal Referring}, a generalized UAV referring task that jointly expands the query modality and the output cardinality. We construct \emph{UniRef-UAV}, a multimodal benchmark that supports text-only, image-only, and text+image queries with modality-dependent target cardinality, where text-only and text+image queries admit no-target, single-target, and multi-target grounding while image-only queries focus on existence-aware single-instance grounding. It also provides in-domain and cross-domain evaluation protocols for visual-query generalization. We further present \emph{UAV-URNet}, a detection-style baseline that maps heterogeneous queries into a shared query space and predicts variable-size target sets through set prediction. Extensive experiments show that UAV-URNet provides a stable and reproducible baseline with more consistent no-target discrimination and a more lightweight, reproducible implementation than large general-purpose MLLMs. Additional domain analysis, query-representation analysis, and ablation studies demonstrate that multimodal queries help reduce visual-query ambiguity and promote a more unified query--target alignment space. The annotations, visual query crops/images, train/validation/test splits, evaluation scripts, and baseline code will be made publicly available to facilitate reproducible research.
