Search papers, labs, and topics across Lattice.
This paper introduces RedFlow, an innovative offline reinforcement learning framework designed to enhance flow-matching Vision-Language-Action (VLA) policies by converting failure experiences into actionable corrections. By implementing a Context-Aware Corrective Matching mechanism and an Adaptive Redirection Objective, RedFlow effectively identifies and redirects failure-inducing actions towards successful alternatives, significantly improving learning efficiency. Experimental results demonstrate that RedFlow increases the real-world success rate of robotic manipulation tasks from 56.7% to 74.7%, outperforming existing offline RL methods and matching the performance of on-policy approaches with significantly fewer training samples.
Redirecting failure into actionable corrections boosts robotic manipulation success rates by nearly 32% while using an order of magnitude fewer training samples.
Flow-matching Vision-Language-Action (VLA) policies have shown strong potential for robotic manipulation but often suffer from compounding errors caused by distribution shifts during deployment. While offline reinforcement learning (RL) provides a practical way to improve deployed policies using rollout data, existing methods either ignore failure data or exploit it only at the trajectory level, resulting in low learning efficiency and persistent errors. We propose **RedFlow**, a fine-grained offline RL framework that redirects failure experiences into action-level corrective supervision for flow-matching VLA policies. RedFlow consists of two key components: (1) a **Context-Aware Corrective Matching** mechanism that identifies failure-inducing actions and retrieves successful alternatives from similar contexts as corrective targets, and (2) an **Adaptive Redirection Objective** that jointly reinforces successful actions, suppresses undesirable ones, and redirects recoverable failures toward corrective targets. By converting both successful and failed experiences into dense supervision, RedFlow enables robust recovery learning from mixed-quality data. Experiments on the LIBERO benchmark and three real-world manipulation tasks show that RedFlow consistently outperforms state-of-the-art offline RL baselines, improving the real-world success rate from 56.7% to 74.7%. It also matches strong on-policy methods (PPO, GRPO, and DDPO) while requiring roughly an order of magnitude fewer training samples.
