Search papers, labs, and topics across Lattice.
The paper introduces Rule2DRC, a large-scale benchmark for evaluating LLM agents in synthesizing DRC scripts from natural language rules, comprising 1,000 tasks and 13,921 evaluation layouts. Unlike prior work, Rule2DRC assesses functional correctness by executing generated scripts on layouts, eliminating the need for labeled test layouts as input to the agent. The authors also propose SplitTester, an execution-guided test case generation method that improves program selection by creating discriminative tests to distinguish between candidate scripts.
LLMs can now be rigorously tested on their ability to generate correct chip design rule checking (DRC) scripts, thanks to a new benchmark that scores scripts based on execution, not just code similarity.
Manufacturable chip layouts must satisfy thousands of geometry-based design rules, and design rule checking (DRC) enforces them by running executable DRC scripts on layouts. Translating natural language rules into correct DRC scripts is labor-intensive and requires specialized expertise, motivating LLM agents for DRC script synthesis and debugging. However, existing benchmarks have small evaluation sets and often evaluate scripts by code similarity rather than execution correctness, and prior machine learning-based methods either ignore execution feedback or require labeled test layouts as agent's input. To this end, we introduce Rule2DRC, a large-scale benchmark for DRC script coding agents with 1,000 rule-to-script tasks and 13,921 evaluation chip layouts for execution-based scoring. Rule2DRC provides an evaluation pipeline that measures functional correctness via DRC execution outcomes without requiring evaluation layouts as input to the agent. We also propose SplitTester, a tester agent for program selection that uses execution feedback to generate discriminative test cases and separate previously indistinguishable candidate scripts, substantially improving Best-of-N selection performance in this domain. We release the code at https://github.com/snu-mllab/Rule2DRC.
