Search papers, labs, and topics across Lattice.
This paper introduces SWAM (Spatial-perceiving World Action Model), a novel framework for embodied navigation that integrates goal intent with trajectory synthesis in a single-pass inference process. By generating intermediate RGB-D sequences alongside action trajectories, SWAM addresses the limitations of existing world model-based planners, such as candidate dependence and computational inefficiency. Experimental results demonstrate that SWAM not only surpasses state-of-the-art two-stage planners in success rate and trajectory accuracy but also exhibits strong zero-shot generalization to new environments.
SWAM achieves superior navigation performance by seamlessly integrating observation and action generation, significantly enhancing efficiency and accuracy in embodied tasks.
Existing world model-based planners for visual navigation typically follow a verification-centric paradigm, decoupling goal intent from trajectory synthesis. This approach suffers from candidate dependence, heavy computational overhead, and inconsistencies between sampled actions and predicted visuals. To address these issues, we propose SWAM (Spatial-perceiving World Action Model), a task-centric joint observation-action generation framework. Given start and goal RGB observations, SWAM performs single-pass inference to simultaneously generate intermediate RGB-D sequences and corresponding action trajectories, promoting goal-consistent trajectory generation and improved spatial feasibility. While SWAM leverages depth pseudo-labels during training to internalize spatial priors, it requires only monocular RGB input at inference time. We further introduce a visual-guided action refinement module and a trajectory-scale regularization loss to enforce fine-grained alignment between motion and visual cues while stabilizing predictions across varying distances. Extensive experiments show that SWAM significantly outperforms state-of-the-art two-stage planners in success rate, trajectory accuracy, and inference efficiency, while demonstrating robust zero-shot generalization to unseen environments.
