Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper addresses the sample inefficiency of vision-based RL for robotic manipulation by bridging the train-test modality gap between privileged simulation states and real-world RGB-D observations. The proposed FOCUS framework, a single-stage PPO, trains the critic on privileged states while dynamically switching the actor's rollouts between RGB-D and privileged-state latents based on KL divergence of action distributions. This approach significantly improves test success rates and learning efficiency across five manipulation tasks, achieving a 5.0x improvement in learning efficiency on Pick-and-Place.
Robots learn faster and succeed more often by intelligently switching between high-fidelity simulation data and real-world vision during training.
Vision-based reinforcement learning for robotic manipulation is sample-inefficient because RGB-D observations are high-dimensional and noisy. Privileged state information available in simulation can accelerate training, but its absence at test time creates a train-test modality gap. We propose FOCUS, a single-stage PPO framework that trains the critic on privileged state while automatically regulating whether the actor collects rollouts from RGB-D or privileged-state latents. Regulation is driven by the KL divergence between the action distributions induced by the two modalities, while representation alignment encourages consistent action selection across them. Together, these mechanisms limit RGB-D rollouts when the actor's action distributions from RGB-D and privileged-state latents disagree. As they align, RGB-D exposure increases, shifting on-policy training toward the RGB-D inputs used at test time. Across five manipulation tasks, FOCUS raises average test success from 0.71 to 0.93 relative to the strongest RGB-D-at-test baseline on each task. When accounting for each method's complete training pipeline, budget-normalized training-success AUC increases from 0.47 to 0.65. On Pick-and-Place, test success rises from 0.47 to 0.86, while AUC increases from 0.12 to 0.61, a 5.0x improvement in learning efficiency over the fixed interaction budget.