Search papers, labs, and topics across Lattice.
PhysCaP is a Physics-Informed Code-as-Policy agent designed for active perception in robotic manipulation, addressing the limitations of vision-language-action policies that rely solely on passive observation. By integrating a physics-informed exploration layer, PhysCaP enables robots to actively seek information about latent physical properties like mass and stiffness through interaction, without the need for additional sensors. Evaluations demonstrate that PhysCaP outperforms existing baselines in real-world manipulation tasks by achieving comparable results with fewer interactions and reduced execution time, highlighting the effectiveness of its dual-agent design for efficient exploration.
PhysCaP enables robots to actively infer hidden physical properties, achieving superior manipulation efficiency without additional sensors.
We present PhysCaP, a Physics-Informed Code-as-Policy agent for active perception in robotic manipulation. While vision-language-action policies excel at imitating demonstrations, they rely on passive observation and fail to infer latent physical properties critical for manipulation. PhysCaP augments code-as-policy frameworks with a physics-informed exploration layer that enables explicit information-seeking through interaction. It introduces training-free physical property extraction modules that estimate object mass and stiffness from robot proprioception without additional sensors. To balance exploration costs and the efficiency of information obtained, PhysCaP employs a dual-agent design: a Planner that decides when to explore and when to stop, and a Prioritizer that filters implausible interactions and ranks the remainder using a heuristic priority score, enabling efficient, targeted exploration. We evaluate PhysCaP on real-world tabletop manipulation tasks (searching for hidden objects, detecting empty cans, and finding ripe avocados) and a simulated task in LIBERO. The results show that existing passive and naive interactive baselines either fail when physical properties are hidden or over-explore, whereas PhysCaP achieves comparable performance with fewer interactions and reduced execution time. Ablation studies further validate the effectiveness of the proposed physical property extraction modules. Project page: https://physcap.github.io
NVIDIA