Search papers, labs, and topics across Lattice.
The PAC-MAN framework integrates perception-aware control barrier functions with reinforcement learning to enhance safety in humanoid dodgeball by utilizing depth data from a head-mounted camera. Evaluated in both controlled and real-world environments, the policy achieves a 95% success rate in evading thrown balls, demonstrating that even limited perception can yield effective evasive maneuvers. The findings reveal that the performance of control barrier structures is heavily influenced by the quality of perceptual input, with the Joint-CBF approach outperforming others under optimal conditions.
A humanoid robot can dodge 95% of thrown balls using only depth data from a head-mounted camera, showcasing the potential of perception-aware safety in dynamic environments.
We present PAC-MAN, a perception-aware CBF-RL framework that couples control-barrier safety with deployment-realistic onboard sensing for whole-body humanoid dodgeball. The deployed policy sees the ball only as segmentation-masked depth from a head-mounted camera, while training-time CBF guidance represents clearance to every body link, and an adversarial motion prior regularizes the resulting evasive reflexes. We evaluate on a controlled any-link contact benchmark with seeded throws in two regimes: single throws and a deployment loop in which the robot walks back to its station and recovers between throws. On this benchmark, the policy comes within a few points of a privileged state oracle: a fixed onboard camera alone is adequate for evasion. We find that usable barrier structure depends on perceptual observability: Joint-CBF gives the best performance with accurate ball states, degrades under fixed-camera observations when used only as training guidance, and recovers with a ball-tracking gimbal or privileged runtime filter. We therefore deploy a lightweight Link-CBF policy zero-shot on the Unitree G1 in the real world, where it tolerates imperfect perception, succeeds on 95% of throws, and uses semantic segmentation to dodge different balls.
