Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
TLC-DiT integrates explicit, task-aligned local visual condition maps into Multitask Diffusion Transformers to resolve policy fragility and opacity caused by implicit visual reasoning. By modulating frozen DINOv2 patch features with CLIP language embeddings via FiLM and CoordConv adapters, the framework exposes interpretable spatial affordances directly to the action generation process. The method achieves a 93.5% success rate on the LIBERO benchmark and doubles real-world bimanual completion rates from 44% to 89% while significantly boosting robustness to visual perturbations.
Robot manipulation policies don't require architectural overhauls to survive visual distribution shifts—explicitly injecting language-modulated local feature maps into standard diffusion transformers doubles real-world bimanual task success.
Language-conditioned robot policies have made clear progress in multitask manipulation, but task-relevant local visual evidence usually stays hidden inside a visual backbone or attention layers. This leaves the policy difficult to inspect and fragile under visual change, two symptoms of a missing explicit, task-aligned local visual channel. We present TLC-DiT, a plug-in extension of the Multitask Diffusion Transformer (DiT) policy that adds explicit task-guided local visual feature maps without changing the diffusion objective or the action-generation process. For each camera view, frozen DINOv2 patch features are modulated by the CLIP task embedding through FiLM and refined by a lightweight CoordConv CNN adapter into smooth spatial maps, which are concatenated with the original global image, language, joint-state, and timestep conditions. On LIBERO, TLC-DiT reaches a 93.5% average success rate, compared with 86.5% for Multitask DiT and 79.25% for SmolVLA. On LIBERO-plus, the total success rate improves from 54.07% to 57.24%, with larger gains under camera, background, and sensor-noise changes. In real-world bimanual tasks, TLC-DiT raises Teabag Putting completion from 44% to 89% while maintaining comparable Match Box Opening performance. Feature-map visualizations confirm that the model attends to task-relevant regions across views and perturbations, providing a direct way to inspect the visual evidence.