Search papers, labs, and topics across Lattice.
This paper addresses the limitations of the DialNav framework for embodied agents by introducing the RAINbow dataset, which significantly expands the training data from 2K to 238K episodes through an automatic generation pipeline. The authors implement Dual-Strategy Training and a localization model to enhance the integration of dialog and navigation, leading to substantial improvements in success rates on both seen and unseen validation splits. As a result, the proposed methods establish a new state of the art in dialog-driven navigation tasks, demonstrating the effectiveness of large-scale data augmentation in this domain.
Expanding training data from 2K to 238K episodes boosts dialog-driven navigation success rates by up to 100%.
For embodied agents capable of physical interaction, the capability to create and understand dialog is crucial to ensure both safety and effectiveness. While DialNav~\cite{han2025dialnav} provides a framework for holistic evaluation of the dialog--execution loop in photorealistic indoor navigation, its performance remains limited by a critical scarcity of training data (2K episodes). To address this, we propose an automatic generation pipeline, and construct the \textbf{RAINbow} dataset, a large-scale training dataset with 238K episodes for DialNav. Our pipeline converts existing VLN datasets into multi-turn dialog and creates cost-efficient and high-quality dataset. Then, we introduce two additional complementary advances to unlock the data's full potential: (1) Dual-Strategy Training, a navigation training scheme to align the navigation training with the dynamic dialog-navigation loop, and (2) a localization model that leverages VLN knowledge. By combining these complementary solutions, our model substantially outperforms the baseline in success rate on both \textbf{Val Seen} (58.24, \textbf{+89\%}) and \textbf{Val Unseen} (29.05, \textbf{+100\%}) splits, establishing a new state of the art.
