HW-Bench (@hardwarebench) header
HW-Bench (@hardwarebench) profile picture

The first hardware benchmark built in partnership the world’s most advanced hardware companies, created by @arenaphysica 💡

New York, USA
Joined September 2026
Photo by HW-Bench (@hardwarebench)Photo by HW-Bench (@hardwarebench)Photo by HW-Bench (@hardwarebench)Photo by HW-Bench (@hardwarebench)
HW-Bench is the first hardware engineering benchmark built in partnership with the world's most advanced hardware companies. HW-Bench evaluates frontier model performance on production hardware data, captures the full hardware context across tools and file formats, and grades every task against rubrics and golden answers reviewed by subject matter experts at partner companies (more to come soon). Our launch results cover the performance of 15 frontier models on designing a 2.45 GHz antenna, and we’re building toward jobs that span disciplines, like designing, manufacturing, and testing a 140 GHz, 512-element D-band phased array. Learn more at hw-bench.com! Created by the team @arenaphysica ⚡
6
4
11
45
9,966
GPT-6.2 Astra and GPT-6.2 Sol enter Hardware Bench at 4th and 5th place. GPT-6.2 Astra ties with GPT-6 Astra on mean accuracy (0.81) but trails it on Pass¹ (53% vs. 67%). GPT-6.2 Sol ties GPT-6.1 Sol on Pass⁵ (33%) but trails it on Pass¹ (50% vs. 57%). See how all the models compare at hw-bench.com!
1
1
7
888
RT @PratapRanade: Fable 5.2 climbing the capability ladder for hardware, just before ANT IPO. But, still much worse on cost and latency tha…
This quoted post is unavailable.
1
1
HW-Bench retweeted
Fable 5.2 joins GPT-6 Astra and GPT-6 Sol on both the cost and latency frontiers. However, it’s the most expensive and slowest model, at $120 and 7 hours per job run. See how every model compares at hw-bench.com!
4
1
4
567
Fable 5.2 joins GPT-6 Astra and GPT-6 Sol on both the cost and latency frontiers. However, it’s the most expensive and slowest model, at $120 and 7 hours per job run. See how every model compares at hw-bench.com!
4
1
4
567
Claude Fable 5.2 is now the most accurate model on @hardwarebench 👀
Claude Fable 5.2 is now the most accurate model on HW-Bench. Its mean accuracy score is 0.845, 0.04 above GPT-6 Astra’s 0.805, and the two models tie on Pass¹ at 67%. See how all the models compare at hw-bench.com!
1
2
37
1,567
Claude Fable 5.2 is now the most accurate model on HW-Bench. Its mean accuracy score is 0.845, 0.04 above GPT-6 Astra’s 0.805, and the two models tie on Pass¹ at 67%. See how all the models compare at hw-bench.com!
1
3
1
31
2,258
HW-Bench retweeted
5 Thanks to @thinkymachines for the AI credits that power this research. Early results, more data coming. Code lands in the repo in a few days after review. Inspired by @arenaphysica @hardwarebench
2
3
315
HW-Bench retweeted
Finally, a benchmark for hardware engineering. Can models do tolerances yet, or still vibes?
2
3
148
What the antenna results show: 1️⃣ 11 of 15 models never passed every critical requirement on all five runs of any task. GPT-6 Astra did it on four of six. Hardware can only ‘compile once,’ so a model that’s only right some of the time can cost a full fab cycle. 2️⃣ No single model is best. Six different models had the top accuracy score on at least one of the six tasks, and sending each task to its most accurate model scored 0.875, vs. 0.805 for GPT-6 Astra on every task, at 73% less cost. 3️⃣ Driving simulation and EDA tools is the hardest part. Only 13% of runs on the HFSS and KiCad tasks passed every critical criterion, vs. 41% on the text and code tasks. 4️⃣ Open-weight models trail. GLM-5.3 and Kimi K3 ranked 12th and 15th by accuracy, yet cost more than twice as much as GPT-6.1 Sol, the second most accurate model.
Today, we're releasing @hardwarebench, the first hardware engineering benchmark built in partnership with the world's most advanced hardware companies. Measurement has always been core to our work at @arenaphysica: the very first version of Atlas shipped with our internal framework for evaluating frontier models on real hardware engineering tasks. HW-Bench evaluates frontier model performance on production hardware data and captures the full hardware context across tools and file formats. Each task is graded against rubrics and golden answers reviewed by subject matter experts at our partner companies, who build some of the most advanced hardware in the world, from the chips behind modern supercomputing to the autonomous systems and vessels redefining defense at sea and beyond (more coming soon). We'll be building towards jobs that span disciplines, like designing, manufacturing, and testing a 140 GHz, 512-element D-band phased array. To start, see how 15 frontier models performed at designing a 2.45 GHz antenna at hw-bench.com.
1
4
1
24
1,747
HW-Bench retweeted
nice!
2
3
158
HW-Bench retweeted
love this. we did the machining side: teapawt.org/MerkleAnders/status/21…
Introducing RealShop-CAD, a benchmark for measuring how well AI turns real production drawings into parts a machine shop can actually make. Built from anonymized production drawings from MES customers, RealShop-CAD includes 228 drawing-to-STEP tasks, graded to 0.1 mm against each part's production 3D model.
1
1
4
185
HW-Bench retweeted
This is huge! Whats harder to benchmark: the models or the engineering workflows ?
1
1
2
56
LLMs still struggle with real engineering tools. On HFSS/KiCad tasks, only 13% of runs passed every critical check vs 41% on text tasks. On PCB layout, just 3 of 75 runs passed, and most set the wrong board stackup.
2
2
1
5
227
HW-Bench retweeted
Arena Physica launched HW-Bench, the first hardware engineering benchmark built with top hardware companies. It grades frontier models on production hardware data across real tools and file formats. Software benchmarks are saturated. Hardware exposes models
Today, we're releasing @hardwarebench, the first hardware engineering benchmark built in partnership with the world's most advanced hardware companies. Measurement has always been core to our work at @arenaphysica: the very first version of Atlas shipped with our internal framework for evaluating frontier models on real hardware engineering tasks. HW-Bench evaluates frontier model performance on production hardware data and captures the full hardware context across tools and file formats. Each task is graded against rubrics and golden answers reviewed by subject matter experts at our partner companies, who build some of the most advanced hardware in the world, from the chips behind modern supercomputing to the autonomous systems and vessels redefining defense at sea and beyond (more coming soon). We'll be building towards jobs that span disciplines, like designing, manufacturing, and testing a 140 GHz, 512-element D-band phased array. To start, see how 15 frontier models performed at designing a 2.45 GHz antenna at hw-bench.com.
3
3
219
Finally someone working on non-RTL benchmarks! now we need HIL testing
Today, we're releasing @hardwarebench, the first hardware engineering benchmark built in partnership with the world's most advanced hardware companies. Measurement has always been core to our work at @arenaphysica: the very first version of Atlas shipped with our internal framework for evaluating frontier models on real hardware engineering tasks. HW-Bench evaluates frontier model performance on production hardware data and captures the full hardware context across tools and file formats. Each task is graded against rubrics and golden answers reviewed by subject matter experts at our partner companies, who build some of the most advanced hardware in the world, from the chips behind modern supercomputing to the autonomous systems and vessels redefining defense at sea and beyond (more coming soon). We'll be building towards jobs that span disciplines, like designing, manufacturing, and testing a 140 GHz, 512-element D-band phased array. To start, see how 15 frontier models performed at designing a 2.45 GHz antenna at hw-bench.com.
1
3
5
459
HW-Bench retweeted
Arena Physica just launched HW-Bench, a benchmark that tests AI models on real hardware engineering. Here's what you need to know. HW-Bench is billed as the first hardware engineering benchmark built with the world's most advanced hardware companies. It uses production hardware data and captures the full hardware context across tools and file formats. Every task is graded against rubrics and golden answers reviewed by subject matter experts at partner companies, with more partners coming. Arena Physica says measurement has always been core to its work, and the first version of its Atlas product shipped with an internal evaluation framework. The launch results cover one job, designing a 2.45 GHz antenna, split into six tasks. The tests ran in HFSS and KiCad. Fifteen frontier models were scored on accuracy, cost and latency. The scoring is strict. Pass^k is the chance that k runs of the same task all pass every critical criterion, so failing one critical criterion fails the run. The team says the next goal is jobs that span disciplines, like designing, manufacturing and testing a 140 GHz, 512-element D-band phased array. They are also asking engineers, companies with designs or data, and researchers to contribute. Rankings may shift as more jobs are added. Top of the leaderboard: - 1st: OpenAI GPT-6 Astra, 67% pass rate, 0.81 accuracy, $50.49 per job, 1h 28m - 2nd: GPT-6.1 Sol, 57% pass1, 42% pass3, 33% pass5, 0.78 accuracy, $8.42, 1h 45m - 3rd: GPT-6 Sol, 37% pass1, 0.71 accuracy, $8.07, 53m - 4th: Anthropic Claude Opus 5, 33% pass1, 3% pass3, 0.71 accuracy, $28.87 - Claude Fable 5.1: 23% pass1, $33.84. Claude Sonnet 5.5: 20% pass1, $5.75 - GPT-6 Luna: 30% pass1 at just $0.55 per job One contributor says no single model wins. Routing each task to its best model scored 0.875 accuracy versus 0.805 for the top single model, at 73% lower cost. Other voices say teams should build benchmarks on their own data, and that benchmark results shouldn't be trusted blindly. HW-Bench launched on Nov 5, 2026, at hw-bench.com.
HW-Bench is the first hardware engineering benchmark built in partnership with the world's most advanced hardware companies. HW-Bench evaluates frontier model performance on production hardware data, captures the full hardware context across tools and file formats, and grades every task against rubrics and golden answers reviewed by subject matter experts at partner companies (more to come soon). Our launch results cover the performance of 15 frontier models on designing a 2.45 GHz antenna, and we’re building toward jobs that span disciplines, like designing, manufacturing, and testing a 140 GHz, 512-element D-band phased array. Learn more at hw-bench.com! Created by the team @arenaphysica ⚡
3
3
6
1,043
hardware has been the blind spot in most benchmarks, good to see someone go after it properly
3
3
209
HW-Bench retweeted
Cool new LLM model benchmark just dropped!
HW-Bench is the first hardware engineering benchmark built in partnership with the world's most advanced hardware companies. HW-Bench evaluates frontier model performance on production hardware data, captures the full hardware context across tools and file formats, and grades every task against rubrics and golden answers reviewed by subject matter experts at partner companies (more to come soon). Our launch results cover the performance of 15 frontier models on designing a 2.45 GHz antenna, and we’re building toward jobs that span disciplines, like designing, manufacturing, and testing a 140 GHz, 512-element D-band phased array. Learn more at hw-bench.com! Created by the team @arenaphysica ⚡
1
2
5
390
One of the most fun projects I've worked on is live: HW-Bench, 15 models on a real antenna design job in HFSS and KiCad. No single model wins: routing each task to its best model beat the top model alone, 0.875 vs 0.805 accuracy at 73% lower cost. hw-bench.com
HW-Bench is the first hardware engineering benchmark built in partnership with the world's most advanced hardware companies. HW-Bench evaluates frontier model performance on production hardware data, captures the full hardware context across tools and file formats, and grades every task against rubrics and golden answers reviewed by subject matter experts at partner companies (more to come soon). Our launch results cover the performance of 15 frontier models on designing a 2.45 GHz antenna, and we’re building toward jobs that span disciplines, like designing, manufacturing, and testing a 140 GHz, 512-element D-band phased array. Learn more at hw-bench.com! Created by the team @arenaphysica ⚡
2
4
13
557
RT @_i_am_arya: Our Why’s behind @hardwarebench at @arenaphysica: - Verifiable, by design…get your judges out of here - Multi-discipline,…
This quoted post is unavailable.
1
6