Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
To prevent language-driven manipulation from overconstraining continuous controllers with fully specified target poses, this paper presents a typed semantic-to-geometric interface that compiles qualitative relational instructions while explicitly preserving task-unspecified degrees of freedom. The compiler grounds relational specifications into continuous task maps and consistent differentials using conformal geometric algebra, composing multi-phase execution through RMPflow. Decoupling semantic topology from geometric grounding cut critical interface errors from 58/60 to 19/60 compared to prompting for full geometric specs, while closed-loop MuJoCo ablations proved that artificially fixing preserved null coordinates worsened median terminal progress error by 20.24–71.00 mm.
Forcing language models to output rigid target poses needlessly overconstrains continuous controllers, whereas compiling semantic relations into differential task maps that preserve geometric null spaces prevents up to 71 mm of terminal execution error.
Natural-language manipulation instructions specify qualitative relations, whereas continuous controllers require state-evaluable task quantities, differentials, and completion conditions. Because a qualitative relation generally leaves part of the relative configuration unspecified, expanding it into a complete pose can introduce unintended constraints. We present a typed semantic-to-geometric interface in which language specifies entities, relations, and phases, while each relation indexes a registered specification of its task-relevant distinctions and preserved freedoms. A robot-side compiler grounds these specifications, constructs relation-specific task maps and consistent differentials using conformal geometric algebra, and composes the resulting policies through RMPflow. To evaluate the division of responsibility between the language model and the compiler, we compared a Semantic Topology interface with one that additionally requires relation-specific geometric specifications over 60 instructions. Both produced correct shared semantic content in 41/60 cases, but critical errors under their respective interface requirements occurred in 19/60 and 58/60 cases. Across 64 grounded evaluations spanning eight geometric relation forms, the task maps preserved registered null directions and responded to relation-relevant perturbations; analytic directional derivatives agreed with finite differences, and Jacobian ranks matched the registered dimensions. In three closed-loop ablations using a simulated Franka Emika Panda in MuJoCo, fixing a relation-preserved coordinate increased median terminal progress error by 20.24--71.00~mm while the retained relation errors remained within their evaluation bounds. These results support compiling relation-visible geometry and preserved freedom together into composable continuous objectives.