Search papers, labs, and topics across Lattice.
This paper introduces FluxBench, a comprehensive benchmarking framework for evaluating AI agents across end-to-end electronic design automation (EDA) workflows, utilizing unified prompts and tool environments. The study reveals significant performance disparities among different agent architectures, with gaps reaching up to 86.27%, and highlights the importance of both agent design and foundational model capabilities in achieving effective automation. Notably, the results demonstrate that FluxEDA outperforms Claude Code by up to 8.39 times in the RTL-to-GDS flow, underscoring that domain-specific skills alone are insufficient for optimal performance in complex EDA tasks.
Performance gaps of up to 86.27% among AI agents in EDA workflows reveal that architecture matters more than just domain-specific skills.
LLM-driven agent systems have emerged as a promising paradigm for electronic design automation (EDA), demonstrating strong potential for automating complex design workflows. However, existing evaluations primarily examine individual language models on isolated EDA tasks, providing limited insight into how different agent systems perform across complete EDA flows. In this work, we present FluxBench, a systematic evaluation of AI agents on end-to-end EDA workflows under unified prompts, tool environments, and technology library settings. Our evaluation covers representative scenarios, including RTL generation with open-source toolchains and an RTL-to-GDS flow using closed-source commercial EDA tools for industrial applications. Through these workflows, we assess agents'capabilities in RTL code generation, iterative repair, tool-feedback utilization, logic synthesis, placement and routing (P&R), and Engineering Change Order (ECO) automation. To further characterize the efficiency of agent systems, we introduce Token ROI, a cost-efficiency metric that measures effective improvements in EDA artifacts relative to token usage and runtime cost. Experimental results show that, even when built on the same foundation model, different agent system architectures can exhibit performance gaps of up to 86.27%. Moreover, among systems with comparable task performance, Token ROI can differ by as much as $105.92\times$. In the RTL-to-GDS flow using PicoRV32 as a case study, FluxEDA achieves an end-to-end score of up to 97.94, outperforming Claude Code equipped with domain-specific EDA skills by up to $8.39\times$. These results indicate that domain-specific skills alone are insufficient to improve agent performance in large-scale EDA scenarios. Instead, both agent system design and foundation model capability play critical roles in enabling effective automated EDA workflows.
