Search papers, labs, and topics across Lattice.
This paper introduces Harness VLA, a memory-augmented framework that enhances frozen Vision-Language-Action (VLA) models by integrating them with a fixed library of analytic primitives for improved manipulation tasks. The approach allows the VLA to operate effectively under deployment perturbations by leveraging task-specific execution traces and failure models, rather than expanding the skill library. Results show significant performance improvements, with up to 38.6 percentage points higher success rates on benchmark tasks compared to existing methods.
Harness VLA boosts the performance of frozen VLA models by 38.6 percentage points on challenging manipulation tasks without the need for finetuning.
Language-conditioned manipulation requires both precise contact-rich control and robust reasoning over language, scenes, and long horizons. End-to-end Vision-Language-Action (VLA) models provide strong local visuomotor skills, but they are trained on in-distribution task trajectories and often fail under deployment perturbations such as semantic retargeting, goal re-binding, spatial-layout shifts, and unstable local contacts. LLM coding agents provide complementary semantic and compositional reasoning, but purely analytic primitives struggle with irregular grasping, constrained placement, and articulated-object interaction. We present Harness VLA, a memory-augmented agentic framework that exposes a frozen VLA as a retryable contact-rich primitive and composes it with a small fixed library of analytic primitives for grounding, staging, transport, navigation, and release. Rather than expanding the skill library, the harness learns the operating range of these fixed primitives from task-specific execution traces, global success rules, and failure models. By lifting semantic re-grounding, non-contact execution, and VLA re-staging to the planner while reserving the frozen VLA for local contact-rich phases, Harness VLA extends pretrained VLAs beyond their original trajectory distribution without finetuning. Across perturbed tabletop, household kitchen, and clean-to-randomized bimanual manipulation, Harness VLA improves over the strongest relevant baselines by 38.6 and 25.4 percentage points on LIBERO-Pro and RoboCasa365, respectively, and reaches 58.4% on RoboTwin C2R.
Tsinghua AI