Existing chip design benchmarks test isolated steps, such as writing RTL from a specification, and are increasingly saturated. ChipAgents is developing Build-A-Chip-Bench with contributions from NVIDIA to the benchmark suite and scoring rubric. The benchmark evaluates AI agents from workload specification through GDS layout, using open-source tools and an open-source PDK to assess their ability to complete the chip-design workflow. Each design must pass a hard gate, covering functional correctness against a hidden verification IP as well as routability, hold timing, and DRC, and designs that pass are scored on PPA against the task’s constraints. The best agent passes the hard gate on 65% of tasks and reaches an overall score of 16.6%. Frontier agents struggle to revisit earlier decisions, protect working behavior while optimizing, and plan around slow tool runs.
Motivation
Existing chip design benchmarks such as VerilogEval, CVDP, and ChipBench have helped bring agents and models to where they are today. They gave the field some of its first shared yardsticks for hardware agents and made results comparable across models. Model scores on each have climbed quickly as a result.
What they measure, though, is narrow. Each is a point test: the agent receives a spec for an isolated block and writes or verifies its RTL. That covers two steps of the flow, RTL and verification, and nothing downstream of them is exercised.
CVDP and ChipBench are both increasingly saturated (Figure 2).
On CVDP, recent models have plateaued at around 65–70% pass@3, and many of the remaining tasks appear to have issues, such as spec ambiguity or malformed testbenches. Further model gains are unlikely to register on this benchmark.
On ChipBench, scores rose steadily with each new model, from 30.7% at release to 90.6% today. With the pass rate now above 90%, that headroom is nearly gone.
The real goal is end to end: taking a design from workload specification all the way to GDS. Point tests leave critical gaps that only appear across the full flow, such as going back to revise an earlier decision or closing PPA. A point test stops once the RTL simulates correctly. But RTL that passes simulation can still miss timing, exceed its area budget, or fail to route, and those problems only show up downstream. Even when an agent handles just one step, its work has to hold up across the rest of the flow, and point tests cannot see whether it does.
From workload specification to GDS
Taking a chip from a workload specification to GDS runs through more than ten major steps, from architecture and microarchitecture through RTL, synthesis, physical design, and signoff. The steps are not independent. A problem often surfaces several steps after the decision that caused it: timing that fails at signoff can trace back to a pipeline choice made during microarchitecture, and routing congestion can trace back to the floorplan.
Functional correctness is only the entry ticket. A design that simulates correctly still has to be built: it must meet its clock target once real wire delays are known, fit its area budget, stay within its power envelope, and route with no design-rule violations. None of this is visible in simulation. It depends on how the logic maps onto physical cells and wires, which is decided across synthesis, placement, and routing, long after the RTL is written.
This makes the flow hard for experienced engineers, and harder still for agents. Doing it well means understanding how the steps relate to each other, going back up the chain to revise an earlier decision when a later step exposes it, and looking forward to anticipate what a choice will cost downstream. Current agents do this only inconsistently.
Build-A-Chip-Bench measures that loop directly. Each task starts from a workload specification and ends with a GDS layout, and the agent owns every step in between.
Task shape
Each task runs in a sandboxed agent container. The agent receives a workload specification and the pinned interface its design must attach to, with open-source tooling and an open-source PDK available inside the container.
Verification lives elsewhere. Hidden test vectors and a hidden VIP run in a separate container that the agent can send requests to but cannot inspect. The grader runs in a third container the agent has no access to, and scores the final submission. It also holds a reference design for each task, which sets a realistic bar: the task’s constraints are calibrated against it, so they are known to be achievable.
Example task
Each task is defined by a task card. In this example, the agent builds an inference accelerator for Qwen3-0.6B INT4, from the pinned checkpoint to a routed GDS. The card is heavily summarized below.
[Summarized from the full task card. Wording condensed; details omitted.]
Design, implement, verify, synthesize, place, and route an accelerator for a pinned Qwen3-0.6B INT4 checkpoint. Keep going until the RTL and post-route netlist pass functional tests, OpenROAD emits a valid GDS, timing closes at 400 MHz, and measured throughput and PPA are reported.
The full architecture and microarchitecture: array shape, tiling, buffering, scheduling, numerics, and the offline weight layout. Only the top-level boundary is fixed, by interface/ABI.md.
Maximize decode tokens per second, after meeting correctness and every area, power, timing, routing, and memory constraint. Do not stop at the first design that passes.
Not provided: held-out tests, expected outputs, a numerical oracle, a physical design, or reference RTL. The run is network-isolated.
Held-out functional tests on a hidden VIP, plus routability, hold-time, and DRC checks, form a hard gate. Qualified designs are then scored on grader-measured decode throughput for the canonical workload.
What the benchmark stresses
Beyond correctness, Build-A-Chip-Bench stresses four capabilities that point tests do not reach.
01
Engineering instinct under a time limit
Each task has a fixed time budget, and synthesis and place-and-route use up most of it, with a single run taking hours. An agent cannot sweep utilization and constraints until timing closes. It has to get pipeline depth, area budget, and clocking roughly right up front, then go straight to the critical path in the timing report instead of rerunning the whole flow, the way an experienced engineer would.
02
Revisiting decisions from tens of hours earlier
Early decisions constrain every step after them, and their consequences can stay hidden until much later in the run. A setup violation at signoff may trace back to a pipeline stage that was too shallow, or congestion to an SRAM partitioning that forces long wires across the die. The agent has to keep track of what it decided and why, and judge whether to restructure the RTL or absorb the fix late in the flow.
03
Complex tool use
The flow chains simulation, synthesis, static timing analysis, and place-and-route in OpenROAD, each with its own scripts, constraints, and report formats. The agent has to write correct timing constraints, tune floorplan and placement parameters, read timing, congestion, and DRC reports, and keep netlists, constraints, and parasitics consistent as they pass from one tool to the next.
04
Physics awareness
Every decision has a physical cost. A wider datapath means more cells and more wire, a poorly placed SRAM macro creates congestion and long nets, and an aggressive clock target forces upsized cells and higher power. The agent has to anticipate these effects when it sets the microarchitecture, places macros, and selects cells, so the final layout closes timing, fits the die, and routes cleanly.
Results
All models are run with OpenCode as the agent.
Each task is scored in two stages. The design first has to pass two gates: functional verification against the hidden VIP, and GDS pass, meaning the design completes the physical flow to a GDS layout that routes, has no hold-time violations, and is DRC-clean. For designs that pass both, we then measure PPA against the task’s constraints, such as throughput, clock target, power, and area. A task scores its PPA vs. constraint if it passes both gates and zero if it does not, and the overall score is the average across all tasks. Because PPA is only measured on designs that pass, strong PPA on a few tasks does not make up for failing the rest (Table 1).
Interesting findings
Below are some observations from the runs so far.
Closed frontier models can work through the full flow
Closed frontier models are generally able to work through the whole silicon design flow, taking a design from the workload specification through RTL, synthesis, and physical design to a GDS layout. On some steps they show promising capability. Backend tool use is reasonably reliable: agents can usually configure and run synthesis and place-and-route tools without much trouble. In some runs, they also make sensible choices between rewriting the RTL and applying a targeted engineering change order (ECO), though this is not consistent. Open frontier models struggle with the frontend and even more with the backend.
Chasing PPA breaks correctness
When agents push for better PPA, they often break something that was already working. In one run, an agent rewrote its datapath to roughly double throughput, introduced a functional bug and a hang in the process, and spent over an hour trying to track down a mismatch of its own making, only to finish about 3% slower than where it started. We see versions of this across many runs, where a correct design is traded away for a performance gain that never materializes and sometimes the run finishes with neither. Agents rarely treat keeping existing behavior intact as a constraint while they optimize.
Physical failures are traced back to RTL too late
When a design misses timing or area, or fails to route, agents almost always respond at the physical level first, re-running place-and-route with different settings. They are slow to go back to the RTL, even when an architectural choice made there is what put the target out of reach. In one run, the agent documented a compact context-storage scheme that would cut SRAM macros from 51 to 27, deferred it, and spent the next six hours trying routing configurations while detailed routing reported hundreds of thousands of violations. It eventually made the architectural change.
Slow tool loops break convergence
Agents bring the habits of fast try-and-retry loops into a flow where a single place-and-route run can take more than four hours. They underestimate how long the slow steps will take, start runs they don’t have time to finish, and often abandon them partway through optimization.
Acknowledgements
Build-A-Chip-Bench is developed in collaboration with industry leaders including (in alphabetical order) Analog Devices, MediaTek, and NVIDIA. We thank them for their technical advice and support.
Citation
Please cite this work as:
@misc{buildachipbench2026,
title = {Introducing Build-A-Chip Bench: From Workload Specification to GDS},
author = {Gunukula, Surya and Wang, Jerry Yicheng and Zhang, Kevin},
year = {2026},
month = {oct},
howpublished = {ChipAgents Research Blog},
url = {https://chipagents.ai/research/build-a-chip-bench}
}