Search papers, labs, and topics across Lattice.
This study introduces PRWeaver, a benchmark designed to evaluate the effectiveness of LLM-based code auditors against long-horizon malicious pull requests, utilizing 208 execution-validated attacks across ten real-world repositories. The findings reveal that traditional commit boundaries do not adequately explain detection evasion, as interleaving and coherent carrier fusion significantly reduce detection rates. Specifically, detection accuracy drops to as low as 16-22% under whole-window reviews, highlighting the critical role of contextual interleaving in the effectiveness of code auditing systems.
Malicious changes can evade detection by up to 22% when interleaved with benign alterations, challenging the reliability of LLM-based code auditors in real-world scenarios.
LLM-based code auditors are increasingly integrated into pull-request (PR) workflows, yet their reliability against adversarial changes distributed across repository evolution remains poorly understood. We introduce PRWeaver, a benchmark of 208 execution-validated attacks from ten real-world repositories, each instantiated under four matched review renderings (832 renderings in total). We evaluate three PR-auditing agents across six auditor-model systems. Across all systems, decomposing an attack changes detection by at most five percentage points, showing that commit boundaries alone do not explain evasion. In contrast, per-PR interleaving at $N=16$ and coherent carrier fusion reduce detection by 5-13 and 10-18 points, respectively. Under whole-window review at $N=24$, detection falls to 16-22%, compared with 50-60% under per-PR review. These results show that access to repository history is insufficient: concealment becomes most effective when benign and malicious changes jointly occupy the auditor's active review context or when the stated purpose plausibly accounts for the attack-bearing diff.
