Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
SWE-Shepherd introduces Process Reward Models (PRMs) to provide dense, action-level supervision for code agents tackling repository-level software engineering tasks. By training a lightweight reward model on SWE-Bench trajectories, SWE-Shepherd guides agents toward higher-reward actions during inference, improving interaction efficiency and action quality. Experiments on SWE-Bench Verified show that PRMs enhance action selection, but aligning intermediate rewards with overall task success remains a challenge.
Step-level reward models can significantly improve the efficiency and quality of code agent actions, but don't guarantee overall task success.
Automating real-world software engineering tasks remains challenging for large language model (LLM)-based agents due to the need for long-horizon reasoning over large, evolving codebases and making consistent decisions across interdependent actions. Existing approaches typically rely on static prompting strategies or handcrafted heuristics to select actions such as code editing, file navigation, and test execution, but they lack fine-grained feedback on intermediate decisions. This leads to inefficient exploration, error propagation, and brittle solution trajectories. To address this limitation, we propose SWE-Shepherd, a framework that introduces Process Reward Models (PRMs) to provide dense, step-level supervision for repository-level code agents. Using trajectories from SWE-Bench, we construct an action-level reward dataset and train a lightweight reward model on a base LLM to estimate the usefulness of intermediate actions. During inference, the PRM evaluates candidate actions and guides the agent toward higher-reward decisions without requiring full reinforcement learning. Experiments on SWE-Bench Verified demonstrate improved interaction efficiency and action quality, while also highlighting challenges in aligning intermediate rewards with final task success.