Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates the impact of goal representation quality on offline goal-conditioned reinforcement learning (GCRL) by systematically corrupting an exact temporal-distance goal representation in deterministic mazes. Contrary to expectations, large degradations in goal representation quality yielded minimal performance changes across OGBench navigation tasks and two algorithms. Instead, corrupting the agent's state representation significantly degraded performance, highlighting it as the primary bottleneck, and suggesting that improved state representations, such as random Fourier positional encodings, offer a more impactful avenue for performance gains.
State representation quality, not goal representation, is the critical bottleneck in offline navigation GCRL, with simple positional encodings boosting performance significantly.
Goal-conditioned reinforcement learning (GCRL) relies heavily on how target goals are represented to the policy. While recent methods encode goals via temporal distance, occupancy, or controllability, it remains unclear how much downstream performance actually depends on representation quality. We study this in offline GCRL by constructing an exact temporal-distance goal representation in deterministic mazes. We then systematically corrupt its geometric quality while keeping the downstream learner fixed. Across OGBench navigation tasks and two algorithms, large changes in goal-representation quality produce almost no change in performance. However, applying the same interventions to the agent's current state more than doubles success, revealing the state pathway as the true bottleneck. Building on this insight, we show that simple random Fourier positional encodings substantially improve performance on the hardest navigation tasks without map information or objective modifications. Overall, our findings suggest that in state-based offline navigation, improving how the agent's current state is represented matters far more than refining the goal representation. Code will be released soon.