Search papers, labs, and topics across Lattice.
Claw-R1 is a novel data middleware system designed to enhance agentic reinforcement learning by managing the entire data lifecycle of agent-environment interactions. It features a Gateway Server that captures multi-turn interactions via a unified LLM API and a Data Pool that organizes these interactions into structured step-level records, facilitating data curation and training preparation. This system emphasizes the significance of treating interaction traces as managed data assets, ultimately improving the efficiency and effectiveness of RL training processes.
Claw-R1 transforms agentic RL by treating interaction data as valuable assets, enabling real-time inspection and curation for optimized training.
Agentic reinforcement learning (RL) has become an important post-training paradigm for turning LLMs from static chatbots into interactive agents, giving rise to representative applications such as OpenClaw. Existing work mainly focuses on policy optimization algorithms and training frameworks, but pays less attention to the full data lifecycle of agent-environment interactions, from data production to training consumption. To bridge this gap, we present Claw-R1, an interactive step-level data middleware system for agentic RL. Claw-R1 connects heterogeneous agent runtimes with RL training backends through two core components: a Gateway Server and a Data Pool. The Gateway Server captures multi-turn interaction steps through a unified LLM API entry point, while the Data Pool organizes them into step-level records consisting of prompt IDs, response IDs, rewards and other metadata. In our demo, users can interactively inspect live trajectories, examine the state, action, and reward of each step, curate data by quality and readiness, and configure training-ready batches for different downstream RL algorithms. Overall, Claw-R1 treats agent interaction traces as managed data assets rather than temporary runtime logs. Through this demonstration, we hope to encourage the community to recognize the importance of data management in agentic RL. Our code is available at https://github.com/AgentR1/Claw-R1 and the demonstration video can be found at link https://youtu.be/Pw47dAOw6B0.
