Search papers, labs, and topics across Lattice.
This paper introduces HAF, a two-part framework designed to adapt generalist vision-language-action (VLA) models for humanoid whole-body loco-manipulation. By employing a hierarchical action-flow generator (HAF-VLA) and a latent offline-to-online reinforcement learning pipeline (HAF-Steer), the framework effectively manages the high dimensionality and interdependencies of humanoid motions while avoiding the computational burdens and safety risks associated with tuning large VLA models. The results demonstrate that HAF significantly enhances whole-body coordination and task performance across seven real-world humanoid manipulation tasks compared to traditional single-stage VLA approaches.
HAF outperforms conventional VLA models in humanoid loco-manipulation by effectively managing complex motion coordination without the need for extensive computational resources.
Humanoid robots hold great promise as general-purpose agents in human-centered environments, yet generalist vision-language-action (VLA) foundation models are not readily applicable to humanoid whole-body loco-manipulation. The high dimensionality and interdependence of humanoid motions make it challenging for conventional single-stage VLA architectures to coordinate locomotion, waist posture, and dual-arm manipulation effectively. Moreover, policies trained through offline behavior cloning can remain suboptimal during real-world deployment. Although online reinforcement learning can refine policies through real-world interaction, directly tuning large VLA backbones demands excessive computation and may introduce safety risks during real-robot exploration. To address these bottlenecks, we introduce HAF (Humanoid Adaptation Framework), a two-part framework consisting of HAF-VLA and HAF-Steer that transfers off-the-shelf generalist VLA foundation models to humanoid whole-body loco-manipulation. HAF-VLA is a hierarchical action-flow generator built on a pretrained flow-matching VLA. It splits full-body action denoising into three sequential stages with stage embeddings and cross-stage KV caches that retain kinematic dependencies, avoiding incoherent whole-body actions from one-shot generation. On top of the frozen HAF-VLA, HAF-Steer is a latent offline-to-online RL pipeline that leverages flow-matching invertibility and DCT-based dimensionality reduction to restrict RL optimization to a compact noise subspace and train a regularized SAC policy. This avoids updating the large VLA backbone and enables efficient real-world policy refinement. Evaluated on seven real-world humanoid loco-manipulation tasks, HAF surpasses vanilla single-stage VLA baselines and improves whole-body coordination and task performance. Project website: https://grange007.github.io/HAF .
