Search papers, labs, and topics across Lattice.
To overcome visual perception failures in poor weather, illumination, and occlusion, the authors introduce the RGBTR-Motion benchmark and SAM-Radar, a multimodal segmentation and tracking framework that grounds direct radar velocity measurements into visual feature space. By treating projected radar returns as foreground supervision and physical persistence cues, SAM-Radar filters clutter without text prompts and maintains track identities across visual dropouts. The approach achieves 0.7027 IoU and 0.8090 F1-50, while outperforming competing methods by 0.2977 MOTA, 0.1603 HOTA, and 0.2857 IDF1.
Direct radar velocity measurements can anchor visual object permanence through complete optical dropouts and clutter, driving a massive 30-point MOTA leap in multimodal moving-object tracking.
Moving-object perception must decide which image regions correspond to real motion and keep every instance identified over time. Methods that read motion from appearance, optical flow, or estimated trajectories lose that evidence under poor illumination, adverse weather, reflections, and occlusion. Radar is a natural remedy because it measures radial velocity directly instead of inferring it from photometric correspondence. However, existing benchmarks do not jointly provide radar measurements, dense moving-instance masks, and temporally consistent identities for surveillance. We therefore introduce RGBTR-Motion, a synchronized and calibrated fixed-camera benchmark that pairs RGB, thermal, and radar streams with dense instance masks and temporally consistent identities across diverse surveillance scenes. We also develop SAM-Radar, an RGB, thermal, and radar-based segmentation and tracking framework built on SAM 3. SAM-Radar's radar-aware detector fuses calibrated RGBT features with radar returns that are grounded at their projected image locations, and motion supervision, implemented as foreground classification of those projected returns, teaches the detector to reject clutter without any text prompt. The tracker associates accepted radar returns with individual trajectories and uses them as physical evidence that a visually degraded target remains present. This allows it to bridge short periods of low visibility or occlusion and reconnect a reappearing target to its existing identity instead of starting a new track. SAM-Radar attains 0.7027 IoU and 0.8090 F1-50, and raises MOTA, HOTA, and IDF1 by 0.2977, 0.1603, and 0.2857 over the strongest competing values.
