Search papers, labs, and topics across Lattice.
The paper introduces OmniRobotHome, a novel room-scale platform designed for real-time multiadic human-robot interaction research in a home environment. It addresses the challenge of occlusion and rapid state changes by using 48 synchronized RGB cameras for markerless 3D tracking of humans and objects, integrated with two Franka Emika robots. Experiments demonstrate that real-time perception and accumulated behavior memory improve safety and enable anticipatory robotic assistance in shared environments.
A 48-camera system finally unlocks real-time, room-scale multi-human, multi-robot interaction research in realistic home environments.
Human-robot collaboration has been studied primarily in dyadic or sequential settings. However, real homes require multiadic collaboration, where multiple humans and robots share a workspace, acting concurrently on interleaved subtasks with tight spatial and temporal coupling. This regime remains underexplored because close-proximity interaction between humans, robots, and objects creates persistent occlusion and rapid state changes, making reliable real-time 3D tracking the central bottleneck. No existing platform provides the real-time, occlusion-robust, room-scale perception needed to make this regime experimentally tractable. We present OmniRobotHome, the first room-scale residential platform that unifies wide-area real-time 3D human and object perception with coordinated multi-robot actuation in a shared world frame. The system instruments a natural home environment with 48 hardware-synchronized RGB cameras for markerless, occlusion-robust tracking of multiple humans and objects, temporally aligned with two Franka arms that act on live scene state. Continuous capture within this consistent frame further supports long-horizon human behavior modeling from accumulated trajectories. The platform makes the multiadic collaboration regime experimentally tractable. We focus on two central problems: safety in shared human-robot environments and human-anticipatory robotic assistance, and show that real-time perception and accumulated behavior memory each yield measurable gains in both.
