Search papers, labs, and topics across Lattice.
This paper introduces RoboHarness, a framework that orchestrates heterogeneous robot policies for long-horizon planning by leveraging multi-modal execution memory to manage capability boundaries and cross-policy mismatches. By utilizing a Memory Bridge to stabilize policy transitions, RoboHarness enables robots to effectively navigate complex tasks without the need for joint retraining of policies. The extensive evaluation across various benchmarks and real-robot scenarios shows significant enhancements in planning efficiency and robustness, particularly in zero-shot contexts and when facing out-of-distribution challenges.
RoboHarness achieves remarkable improvements in long-horizon planning by seamlessly orchestrating diverse robot policies without retraining, even in uncertain environments.
Long-horizon robotic tasks require diverse capabilities that no single policy can reliably provide. Heterogeneous policies offer complementary strengths, but orchestrating them requires reasoning over uncertain capability boundaries and cross-policy distribution mismatch, which are largely overlooked by existing planning methods built on homogeneous, predefined skills with fixed applicability. We propose RoboHarness, a unified framework that encapsulates independently developed robot control systems as reusable agentic skills. Although instantiated in this work with VLAs, RL policies, and task-and-motion planning (TAMP) systems, RoboHarness is designed as a general framework compatible with a broader range of robot policies, such as navigation policies, model predictive controllers, and world-action models. RoboHarness uses multi-modal execution memory and online evidence to characterize policy capability boundaries for capability-aware decomposition and routing. To stabilize policy handoffs, its Memory Bridge retrieves execution trajectories associated with the next policy, estimates its in-distribution state region, and guides the robot toward that region without joint policy retraining. Extensive experiments on three public benchmarks, 500 customized tasks, and 135 real-robot experiments demonstrate effective capability-aware routing and stable policy orchestration, yielding substantial improvements in zero-shot long-horizon planning and out-of-distribution robustness.
