Search papers, labs, and topics across Lattice.
This paper introduces Linear Attention-based Navigation (LANav) for Open-Vocabulary Object Goal Navigation (OVON), addressing the limitations of Transformer-based policies that struggle with context length scalability. By employing a structured state update mechanism through Weighted State-Expansion Linear Attention (WSLA), LANav consistently outperforms Transformer baselines, achieving a 36.4% average success rate on HM3D-OVON and demonstrating significant improvements in long-distance navigation tasks. The results underscore the critical role of state update design in enhancing navigation performance, particularly in real-world applications, where LANav also achieved an 82% success rate in practical trials.
LANav outperforms traditional Transformer-based navigation policies by 6.3 percentage points, revealing the power of structured state updates in complex environments.
Open-Vocabulary Object Goal Navigation (OVON) requires agents to operate under partial observability, making effective internal state updates critical for navigation performance. This update is implemented by the policy network, where recent approaches adopt Transformer-based backbones with self-attention over a context window to integrate temporal information. However, our controlled experiments show that performance does not scale with context length under Transformer-based policies, questioning the suitability of self-attention for state integration in navigation. To this end, we propose Linear Attention-based Navigation (LANav), which adopts linear attention (LA) as the policy backbone to maintain a structured state update rather than self-attention over the context window. Across multiple LA variants evaluated under identical settings, LANav consistently outperforms Transformer-based baselines. Performance improves as state update mechanisms become more structured and regulated, highlighting the importance of state update design. To improve state update effectiveness, we introduce Weighted State-Expansion Linear Attention (WSLA), which expands each attention head's state into multiple sub-states and uses learnable weighted readout to aggregate expanded sub-states. Equipped with WSLA, LANav achieves 36.4% average success rate (SR) on HM3D-OVON, outperforming Transformer-based counterparts by 6.3 percentage points in macro-averaged SR, while maintaining computational efficiency. Distance-stratified results show larger gains in long-distance episodes, while HSSD transfer and fine-tuning demonstrate robustness across scene distributions. Real-world deployment on a Unitree Go2 further achieves an 82% success rate over 50 trials, supporting the practical feasibility and sim-to-real transfer of LANav.
