Search papers, labs, and topics across Lattice.
This paper introduces COMAP, a framework that enables the co-evolution of textual world models and agent policies through closed-loop interaction, allowing language agents to better anticipate environment dynamics and refine their actions. By integrating future state feedback prediction with agent reflection, COMAP adapts the world model to the agent's evolving interaction distribution, enhancing decision-making capabilities. The framework demonstrates significant performance improvements across various benchmarks, including a 16.75% relative gain with Qwen3-4B, highlighting its effectiveness in dynamic environments.
Co-evolving world models with agent policies leads to a 16.75% boost in performance, revolutionizing how language agents navigate complex tasks.
Equipping language agents with world models enables them to anticipate environment dynamics and evaluate candidate actions before execution. However, existing textual world models are typically fixed after training, preventing them from adapting to the on-policy state-action distributions induced by an evolving agent. Meanwhile, agent-improvement methods often rely on external rewards or verifiers, limiting their applicability in realistic interactive environments. In this paper, we propose COMAP, a novel framework that co-evolves textual world models and agent policies through closed-loop interaction. At each decision step, the world model predicts future state feedback for candidate actions, and the agent performs future-aware reflection by estimating the reliability of this feedback and refining its action accordingly. The resulting on-policy trajectories are then used to update the world model via self-distillation, allowing it to better match the agent's evolving interaction distribution. Across embodied task planning, Web navigation, and tool-use benchmarks, COMAP consistently outperforms competitive baselines, e.g., +16.75% relative improvement with Qwen3-4B. Further analyses show that the co-evolutionary loop improves the world model's prediction accuracy over time and leads to more effective long-horizon decision-making. Our code is available at: https://github.com/loyiv/CoMAP.
