Search papers, labs, and topics across Lattice.
HumanFlow, a latent diffusion model, is introduced to unify human motion tracking and forecasting by conditioning on the 3D scene context. This approach addresses the limitations of existing methods in handling occlusions and partial visibility, leading to more plausible and accurate human motion estimates. By tightly coupling HumanFlow's latent space with a flow-matching-based MPC policy, the method achieves superior MAV social navigation performance in simulations with real human trajectories, even under partial observability.
Diffusion models can now drive safer and more efficient robot navigation among humans by unifying motion tracking, forecasting, and control in a single latent space.
Robust and accurate perception of humans in their 3D scene context is essential for integrating robots into everyday environments. Existing approaches, however, often fail to predict plausible and accurate human motion estimates that are consistent with the surrounding scene, especially in the presence of heavy occlusions or partial visibility. This can limit both safety and efficiency for robotic operations. We introduce HumanFlow, a latent diffusion model that unifies human motion tracking and forecasting, conditioned on the 3D scene context. We show that our human motion model produces smooth and accurate predictions under challenging conditions, including heavy occlusions, and outperforms state-of-the-art methods in tracking accuracy while being significantly more efficient. Furthermore, we show how HumanFlow's latent space can be tightly coupled with control by conditioning a flow-matching-based, approximate MPC policy on these representations. We validate our policy in simulation with real human trajectories for MAV social navigation, demonstrating superior navigation performance and remaining collision-free, even under partial observability of the human.
ETH