Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces GeomPrompt and GeomPrompt-Recovery, lightweight modules that adapt frozen RGB-D semantic segmentation models to handle missing or degraded depth data by synthesizing a geometric prompt from RGB or correcting degraded depth, respectively. These modules are trained with downstream segmentation supervision, recovering a geometric prior useful for segmentation without explicitly estimating depth. Experiments on SUN RGB-D show that GeomPrompt improves mIoU over RGB-only inference and GeomPrompt-Recovery enhances robustness under depth corruption, while also being more efficient than monocular depth estimation baselines.
Forget slow depth estimation: GeomPrompt lets you synthesize task-relevant "geometric hints" from RGB to boost RGB-D segmentation when depth data is missing or noisy.
Multimodal perception systems for robotics and embodied AI often assume reliable RGB-D sensing, but in practice, depth is frequently missing, noisy, or corrupted. We thus present GeomPrompt, a lightweight cross-modal adaptation module that synthesizes a task-driven geometric prompt from RGB alone for the fourth channel of a frozen RGB-D semantic segmentation model, without depth supervision. We further introduce GeomPrompt-Recovery, an adaptation module that compensates for degraded depth by predicting the fourth channel correction relevant for the frozen segmenter. Both modules are trained solely with downstream segmentation supervision, enabling recovery of the geometric prior useful for segmentation, rather than estimating depth signals. On SUN RGB-D, GeomPrompt improves over RGB-only inference by +6.1 mIoU on DFormer and +3.0 mIoU on GeminiFusion, while remaining competitive with strong monocular depth estimators. For degraded depth, GeomPrompt-Recovery consistently improves robustness, yielding gains up to +3.6 mIoU under severe depth corruptions. GeomPrompt is also substantially more efficient than monocular depth baselines, reaching 7.8 ms latency versus 38.3 ms and 71.9 ms. These results suggest that task-driven geometric prompting is an efficient mechanism for cross-modal compensation under missing and degraded depth inputs in RGB-D perception.