Heng Yang

Robots can now navigate and manipulate objects based on sound in entirely new environments by fusing acoustic cues with explicit spatial memory derived from egocentric depth observations.

Forcing all memory into policy weights fundamentally bottlenecks robot manipulation: offloading persistent object identity and containment to an external, LLM-queried ledger jumps benchmark success from 45.9% to 64.3%.

This work proposes RoboMP-DINOv2 (Robotics Mask-Prompted DINOv2), a full-scene vision encoder that treats masks as spatial prompts rather than visibility filters, and introduces masked-region color randomization (MCR) to improve appearance robustness, yielding RoboMP-DINOv2-MCR.