Search papers, labs, and topics across Lattice.
This paper introduces KV-Control, a novel parameter-efficient interface for integrating trajectory control into text-conditioned 3D human motion models without compromising the pretrained motion prior. By leveraging a compact attention-side control mechanism, KV-Control allows for precise geometric constraints to be injected into the self-attention layers of frozen transformers, achieving sub-centimeter accuracy in motion tracking. The method retains the quality of text-conditioned motion while significantly reducing the computational burden associated with traditional control methods, thereby enhancing practical animation workflows.
Achieving sub-centimeter accuracy in trajectory-controlled text-to-motion generation without sacrificing the quality of the pretrained model could revolutionize animation workflows.
Text-conditioned 3D human motion models now synthesize plausible motions from prompts, but practical animation and embodied-agent workflows rarely stop at text: a character may need to follow a sketched root path, hit an end-effector target, or satisfy a multi-joint trajectory while still preserving the gait, style, and intent described by language. This exposes a control trade-off. A trajectory controller should be precise without overwriting the pretrained text-conditioned motion prior, yet existing solutions either duplicate large portions of the generator to regain per-layer control access or move much of the cost to test-time optimization. We introduce KV-Control, a compact attention-side control interface for frozen masked text-to-motion transformers. The key idea is to make geometric constraints available as memory inside self-attention rather than injecting them through a global pose token or enforcing them only at the output side. To support this interface, we co-design a part-tokenized motion substrate and controller: \textbf{PartVQ} learns anatomy-aligned part codebooks, T-Concat exposes each frame--part token as an attention-addressable site, and KV-Control injects control-conditioned key/value memories at every self-attention layer while preserving the pretrained query stream, text cross-attention, FFN, and all backbone weights. The resulting adapter adds only trainable injection parameters atop a shared trajectory encoder, yet tracks root and multi-joint constraints with sub-centimeter accuracy under the inherited refinement protocol while retaining text-conditioned motion quality. KV-Control reframes trajectory conditioning as lightweight memory retrieval, providing a small, precise, and transparent control interface for text-to-motion generation.
