Search papers, labs, and topics across Lattice.
This paper explores the implementation of constraint-based oversight for coding agents, addressing the challenges of human review in software development. By applying established management techniques such as access control and strict coding conventions, the authors demonstrate a significant improvement in the detection of security vulnerabilities, achieving a recall rate increase from 54.5% to 90.9% with a constrained oversight system. The findings suggest that these methods not only enhance security but also reduce the cost of human oversight in coding tasks, particularly in languages like Python that lack inherent guarantees.
A constraint-based oversight system can boost vulnerability detection in coding agents from 54.5% to 90.9%, making human review more efficient and secure.
Coding agents are capable; human oversight is the bottleneck. Unconstrained agents introduce security risks, erode codebase scalability, and make human review increasingly costly. We argue that the same methods used for decades to manage large human engineering teams: access control, network policies, strict coding conventions enforced by tooling; transfer directly to coding agents, and are cheaper (in token) than recent agentic scaffolding. We sketch a start-to-end system on this principle, and report a controlled experiment in scalable oversight: a small reviewer (Gemma 4 e4b) inspects a Python codebase containing 11 inserted backdoors. Recall rises from 54.5% (unconstrained, no tools) to 90.9% (constrained substrate plus a ~200-LoC `docs` CLI), with substrate and tools contributing independently. We choose Python deliberately: substrate-level oversight gains are largest where the language gives the fewest guarantees by default; the principles extend to languages like Rust.
