Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces 2D-FET-Bench V2, a novel benchmark comprising 128 tasks for evaluating language model agents' ability to perform flake-specific Field-Effect Transistor (FET) layout construction from optical microscopy data. The benchmark utilizes microscopy-derived flake contours, including complex geometries, and assesses agents' generation of GDSII layouts against deterministic geometric and structural verification. The best-performing GPT5.6-Luna with ReAct-3 achieved 62.3% pass@1, demonstrating significant progress in automated nanoscale device design while highlighting remaining challenges in consistency and expert-validated reliability.
LLM agents can now reliably design complex nanoscale FET layouts from microscopy images, but struggle with consistent, expert-validated execution.
Field-effect transistor (FET) layouts on exfoliated two-dimensional flakes are typically drawn by hand for each flake, placing contacts and gates to match its position and outline in optical micrographs. To our knowledge, no executable benchmark tests whether language-model agents can perform this flake-specific construction reliably. We introduce 2D-FET-Bench V2, a benchmark of 128 layout tasks built from microscopy-derived flake contours, including hole-containing flakes and multi-flake tasks. Each task supplies a textual device specification and contour coordinates. An agent generates typed polygon and path operations rendered to GDSII. A deterministic verifier checks geometric and structural requirements, and a separate integrity check verifies that the supplied contours remain unchanged. Scripted reference layouts pass all 128 tasks, showing that every task is solvable. We evaluate six models and seven workflow and scaffold variants of GPT5.6-Luna, with five attempts per task. The best-performing configuration in the six-model panel, GPT5.6-Luna with ReAct-3, passes 62.3% of attempts and solves 80.5% of tasks at least once (coverage) and 43.8% in all five attempts (consistency). ReAct-3 exceeds the one-pass Plan-and-Execute by 27.0 pass@1 points at 2.46 times the tokens. An expert audit of one sampled verifier-passing layout per covered task, across five ReAct-3 configurations, accepts 56.4% to 63.5% of them. The benchmark evaluates geometric and structural FET layout construction.