Search papers, labs, and topics across Lattice.
This paper introduces Agent-Authored World Modeling (AAWM), a novel training procedure that shifts the focus from next-observation prediction to decision-oriented dynamics by aligning supervision with an agent's specific decision needs. By allowing agents to identify and retrieve relevant transition evidence based on their current state before acting, AAWM synthesizes training targets that better reflect the dynamics crucial for effective decision-making. Experimental results demonstrate that AAWM significantly enhances learning efficiency across various environments compared to traditional next-observation prediction methods.
Decision-aware training signals outperform traditional next-observation predictions, leading to more effective learning in LLM agents.
Recent studies on world modeling for Large Language Model (LLM) agents typically formulate the learning objective as next-observation prediction. However, this objective ties supervision to what a transition happens to reveal, which may omit the dynamics most relevant to the agent's current decision. To bridge this gap, we propose Agent-Authored World Modeling (AAWM), a training procedure that constructs supervision from the policy's own decision needs. Specifically, at each state, the agent identifies what it needs to understand about the environment before acting. These needs drive the retrieval of relevant transition evidence across trajectories, which is then synthesized into training targets that capture decision-oriented dynamics instead of reconstructing the next observation. This aligns the training objective with the dynamics the policy needs before acting, not with the contents of the next observation. Experimental results validate the effectiveness of AAWM across multiple environments and training settings. These results show that decision-aware world-model targets provide a more effective learning signal than next-observation prediction.
