Search papers, labs, and topics across Lattice.
This paper introduces Step-Aware Contrastive Alignment (SACA), a novel framework for Vision-Language Navigation in Continuous Environments (VLN-CE) that addresses the limitations of sparse reward signals in Reinforcement Fine-Tuning (RFT). SACA uses a Perception-Grounded Step-Aware auditor to evaluate progress step-by-step, identifying valid trajectory prefixes and divergence points, enabling dense supervision from imperfect trajectories. Experiments on VLN-CE benchmarks demonstrate that SACA achieves state-of-the-art performance by dynamically routing batches to specialized resampling and optimization strategies based on scenario-conditioned group construction.
Stop letting sparse rewards bottleneck your VLN agent: SACA disentangles failed trajectories into valid prefixes and divergence points for dense supervision, unlocking SOTA performance.
Vision-Language Navigation in Continuous Environments (VLN-CE) requires agents to learn complex reasoning from long-horizon human interactions. While Multi-modal Large Language Models (MLLMs) have driven recent progress, current training paradigms struggle to balance generalization capability, error recovery and training stability. Specifically, (i) policies derived from SFT suffer from compounding errors, struggling to recover from out-of-distribution states, and (ii) Reinforcement Fine-Tuning (RFT) methods e.g. GRPO are bottlenecked by sparse outcome rewards. Their binary feedback fails to assign credit to individual steps, leading to gradient signal collapse in failure dominant batches. To address these challenges, we introduce Step-Aware Contrastive Alignment (SACA), a framework designed to extract dense supervision from imperfect trajectories. At its core, the Perception-Grounded Step-Aware auditor evaluates progress step-by-step, disentangling failed trajectories into valid prefixes and exact divergence points. Leveraging these signals, Scenario-Conditioned Group Construction mechanism dynamically routes batches to specialized resampling and optimization strategies. Extensive experiments on VLN-CE benchmarks demonstrate that SACA achieves state-of-the-art performance.
