Search papers, labs, and topics across Lattice.
This paper introduces WorldDiT, a unified diffusion transformer architecture that integrates action generation with visual world modeling, eliminating the reliance on large pretrained vision-language models. The approach generates continuous action chunks while predicting RGB patch targets from future camera frames, achieving competitive performance across four LIBERO simulation suites. Notably, WorldDiT establishes a strong sub-billion-parameter baseline, positioning it on the Pareto frontier for model efficiency and success rates in robotic control tasks.
WorldDiT achieves robust action generation and visual modeling without the need for bulky pretrained models, setting a new standard for efficiency in robotic policies.
Many recent robot policies pursue stronger control by using large pretrained vision-language models (VLMs) as the action backbone. We introduce WorldDiT, a unified diffusion transformer architecture that couples action generation with visual world modeling and achieves strong performance without a large pretrained VLM action backbone. During training, a single diffusion transformer generates continuous action chunks and predicts normalized RGB patch targets from future camera frames. Across four LIBERO simulation suites, WorldDiT lies on the reported Pareto frontier for total model parameters and mean success among methods reporting all four suites. These results provide a strong sub-billion-parameter baseline for future scaling studies.
