Hilde (@zekiworks) header
Hilde (@zekiworks) profile picture

We build open-source tools for reading and coding, train small models, and publish AI benchmarks.

Zürich, Switzerland
Joined September 2026
Photo by Hilde (@zekiworks)Photo by Hilde (@zekiworks)Photo by Hilde (@zekiworks)Photo by Hilde (@zekiworks)Photo by Hilde (@zekiworks)Photo by Hilde (@zekiworks)Photo by Hilde (@zekiworks)Photo by Hilde (@zekiworks)Photo by Hilde (@zekiworks)Photo by Hilde (@zekiworks)
On the same 763 questions, switching from text to page images costs Gemini Decision 1 9.7 accuracy points, versus 24 on average for GLiDE, Liquid and KnowLine. With text and image together, Gemini leads the decision models tested at 78.4% (KnowLine: 76.0%).
1
3
36
Gemini Decision 1 loses 4.5 accuracy points without Surya OCR 2: 34 fewer correct answers, versus 19 points on average for the other three models shown. With OCR, KnowLine leads these four (75.5%); on page images alone, Gemini leads (66.4%). zekiworks.github.io/analysis…
3
25
On the same 763 questions, switching from text to page images costs Gemini Decision 1 9.7 accuracy points, versus 24 on average for GLiDE, Liquid and KnowLine. With text and image together, Gemini leads the decision models tested at 78.4% (KnowLine: 76.0%).
1
10
Gemini Decision 1 loses 4.5 accuracy points without Surya OCR 2: 34 fewer correct answers, versus 19 points on average for the other three models shown. With OCR, KnowLine leads these four (75.5%); on page images alone, Gemini leads (66.4%). zekiworks.github.io/analysis…
8
REAX 1.3 (57B, two models) scored highest on math and logic among the decision models we tested. KnowLine 31B Gen1.5, half its size, was 13 and 3 answers behind and led English reading. Our paired test can’t separate them. One of REAX’s two models blends in KnowLine 31B Gen1.
🥇 REAX 1.3 is #1 on the Hugging Face Decision Index. We produced the best decision model in the world. 274 models. One top spot. Built on Bittensor $TAO. Four weeks ago, Subnet 92 didn't exist. Then the miners went to work: an open competition, two selected miner models, one post-production run. Now REAX 1.3 sits at the top of Decision Index 0.4.1 with a Full score of 74.0. This is what decentralized AI looks like when it ships.⚡ Live today on a Jev-compatible API decisionmodels.io huggingface.co/spaces/multim…
1
2
2
7
216
On 2,163 math questions, REAX 1.3 got 78.0% right, 13 answers more than KnowLine 31B Gen1.5. The next model, Liquid d1-ultra, is 84 answers behind REAX, a gap that holds up in our paired test. Nearly all these questions are past public exam questions.
1
1
2
47
REAX 1.3 isn’t in Artificial Analysis’s (AA) Decision Index, so we applied AA’s Decision Score, which punishes confident errors, to our 3,288 questions. REAX scores 79.3, KnowLine 31B Gen1.5 78.9. Mammoth-1 (AA #3) is third, ahead of AA’s #1 and #2. zekiworks.github.io/analysis…
1
2
39
Cost per 1M questions: $8.37 for d1-ultra, up to $148 for REAX 1.3. At that upper bound, REAX costs about 18 times as much. REAX’s figure is our GPU’s full running time. AA prices whole tasks, not single questions, so these numbers don’t compare with AA’s.
1
1
3
21
All 16 decision models had a median under half a second. REAX 1.3: 0.20 s on our GPU. d1-ultra: 0.18 s over its API. vLLM-SR d3: 0.06 s on our GPU. AA timed Drex Max v1.1 at 0.28 s per task; we measured 0.33 s per question. All results: zekiworks.github.io/analysis…
1
3
26
Perplexity reported 86% accuracy and a 370 ms median response time on Parallel’s benchmark. Our three runs agree on accuracy. Our median was about 110 ms slower, but we measure from our own machine, so it includes network time. All 36 decision setups without search averaged 47.5%: close to random guessing, and far below always picking the first choice. zekiworks.github.io/analysis…
1
1
7
In Turkish, search sometimes found text from the question itself. Of the 147 questions Perplexity’s Decider got right only with search, 41 shared at least 8 consecutive words with a search result. In English, 0 of 13 did. Decider got 94 right only with search off, leaving a net gain of 53 out of 2,198. A second Turkish run gave a net gain of 52. zekiworks.github.io/analysis…
6
AA’s index puts d1-ultra, SubQ Snap 1.1 and Elephant 9B at 65. On our questions, the tie breaks. English reading: d1-ultra is in a close leading group; SubQ and Elephant trail.
Subquadratic has released SubQ Snap 1.1, tying d1-ultra and Elephant 9B for first on the Artificial Analysis Decision Index v1.0, driven by a low hallucination rate SubQ Snap 1.1 is the new decision model from @subquadratic. It is priced at $0.04 per 1M input tokens and supports a 384K token context window, compared to 32K for d1-ultra and 16K for Elephant 9B. Key takeaways: ➤ SubQ Snap 1.1 scores 65 on the Artificial Analysis Decision Index v1.0, tied with d1-ultra and Elephant 9B: the three models are 8 points ahead of the next model, Jev 1.14 (57). ➤ SubQ Snap 1.1's AA-Omniscience-Decision result is driven by a low hallucination rate: its non-hallucination rate of 86% is second only to Elephant 9B (93%), ahead of d1-ultra (85%). Its accuracy is 20%, compared to 32% for d1-ultra and 12% for Elephant 9B. The non-hallucination rate is the share of questions not answered correctly where the model chose "I don't know". ➤ SubQ Snap 1.1 scores 18% on HLE-MultiChoice: level with Elephant 9B (18%) and ahead of d1-ultra (15%). HLE-MultiChoice is the text-only multiple-choice subset of Humanity's Last Exam. ➤ SubQ Snap 1.1 costs $30 per 1M tasks: comparable to d1-ultra ($32), and ~3.6x the cost of Elephant 9B ($8.4), which scores the same 65.
3
6
1
29
8,306
At ≥95% confidence, d1-ultra kept 549 of 2,198 answers, with no observed errors. It also made none in four further runs. SubQ: 665 kept, 2.7% wrong. Elephant: 393 kept, 3.3% wrong.
1
1
5
127