Search papers, labs, and topics across Lattice.
This paper introduces OpenCodeReview, a deterministic framework designed to enhance the reliability of LLM-based code review agents by addressing the issues of non-determinism and context locality. By implementing a multi-layer rule system for file selection, a curated tool set for grounded review, and a falsification-first filter to eliminate hallucinated comments, OpenCodeReview significantly improves the precision and efficiency of code reviews. Evaluated on AACR-Bench, it achieves up to 2.17x higher SEM-F1 scores compared to existing agents while using 5-15x fewer tokens, demonstrating its effectiveness in real-world scenarios.
OpenCodeReview achieves over double the accuracy of leading LLM code review agents while drastically reducing token consumption.
LLM-based code review agents promise scalable, always-on review, yet current systems suffer from two intertwined weaknesses: (1) non-determinism--unbounded tool use makes review outcomes unstable, and (2) context locality--the reviewer's access remains bounded to the diff, capping discoverable issue depth. Both give rise to three challenges: misaligned context retrieval, a coherence-efficiency trade-off in multi-file pull requests, and hallucinated comments that erode trust. To address these, we introduce OpenCodeReview, built on deterministic engineering for uncertain agents: rather than granting maximal freedom, we inject determinism at three deliberate pipeline points. Rule-Guided Dispatch uses a multi-layer rule system to deterministically select files and review criteria, eliminating variability of agent-driven triage. Grounded File Review replaces free-form exploration with a curated tool set exposed through a ReAct loop, while file-level parallel SubAgents balance context coherence against efficiency and recover cross-file dependencies on demand. Independent Reflection introduces a falsification-first filter under an asymmetric information boundary--the reflector sees only the diff, not the agent's tool-augmented exploration--removing hallucinated comments without self-reinforcing bias, improving precision while preserving recall. On AACR-Bench (200 real-world PRs, 10 languages, 1,505 expert-verified comments), OpenCodeReview outperforms mainstream coding agents (e.g., Claude Code and Codex) across six LLM backends, achieving up to 2.17x higher SEM-F1 (25.10% vs. 11.57%) while consuming 5-15x fewer tokens. We open-source OpenCodeReview at https://github.com/alibaba/open-code-review.
