EEBench measures AI circuit designs with SPICE, BOM cost, and tolerance-corner checks
- EEBench V1 evaluates AI-produced analog and digital circuit designs by building the design, generating a circuit graph and BOM, then running deterministic SPICE simulations and design checks against measured limits.
- In a residential energy-meter task, a circuit must hold a processor rail above 3.0 V for 20 ms after a 5 V supply loss; one shown submission fell below the limit after 0.85 ms.
- The benchmark uses real manufacturer components and datasheet-derived specifications, testing effective capacitance, voltage ratings, package, dielectric, tolerance corners, recovery behavior, availability, and cost rather than ideal textbook values.
- EEBench combines technical results with BOM cost efficiency against a reference bill of materials, but cost improves a score only after the circuit passes its functional requirements.
- On the September 1 leaderboard, Claude Opus 5 scored 61.6% across 13 EEBench V1 tasks, while Grok 4.6 scored 57.1%; V1 does not yet test PCB layout, manufacturing, or hardware bring-up.
Hacker News opinions
I am interested in Astra's PCB routing, but sourcing from DigiKey or LCSC, finding substitutes, reading datasheets, and extracting simulation models are the real pain points.
I think routing is the easy part. Autorouters have existed for a long time, while component placement and iterative placement changes are much harder.
Supply-chain substitution is why I care about this. Formally verifying that an alternate part still meets the original constraints would be useful, and LLM datasheet extraction is starting to reason about curves and footnotes.
I know Schematik, which is working on BOMs, schematics, and firmware first, then may expand into circuit boards. Its early Astra experiments look promising.
AI has helped me sort libraries, draw footprints, manage BOMs, and review work. I still draw schematics and lay out boards because it makes too many stupid errors, though it does catch mistakes in my reviews.
I tried this repeatedly in 2024 and 2025. Recent models can make mostly competent schematics for familiar parts if I provide datasheets and errata, but PCB layout remains terrible, and analog or RF falls apart quickly.
AI is useful for talking through component choices, but every attempt to have it design a board has failed spectacularly for me. It confidently produces nonsense when asked for a print-ready 3D model too.
If a model gets a camera and a test jig, I think it may inspect physical prototypes and iterate against real-world feedback rather than simulation alone.
I got an LLM to write a camera-capture and Bezier-warp script for an e-ink display, then left it overnight to confirm full control by capture round trips. It made meaningful progress in a couple of hours after I guided the setup.
Give an AI a real solver and business constraints, and it can do useful work mapping rules into constraints, balancing costs, and building deterministic and probabilistic acceptance tests. EEBench's use of atopile's constraint solver is the interesting part.
I think we are less than a year from a prompt producing a full assembly and enclosure. PCB routing has to account for power traces, ground loops, overlap, RF practices, and more than simply joining wires.
I have taken a Codex and KiCad MCP flex PCB through JLC and PCBWay DRC validation, but have not ordered or programmed it yet. Another person reports getting simple JLCPCB boards fabricated, assembled, and working, but warns that the missing 10 percent is rough if you cannot verify the result yourself.