i love it when the best open weight model in the world (kimi k3.1 pro) randomly starts speaking another language and ramble about mouses and hospitals in the middle of a security audit

Kimi K3.1 Pro (max) on black hole benchmark 1.0 (2.0, which is much harder, is running now) This isn’t bad, it isn’t great. Its just “good”. Below sonnet 5.5 for sure I’d put it in between GPT 6.1 Sol and GLM 5.3

plush :D

True

Visualizing AA in 3d Currently the closest to the ideal corner is haiku 5.5

210 times cheaper, 19 times faster, and a good chunk higher accuracy than jev on this task It’s the power of specialization

i didn’t know a human is more expensive than sol per task when measuring the size of a box thank you @ArtificialAnlys credit to @morriszdweck for finding this

this is better

opus 5.5 see all results: bhbench.ssh.codes

Gemini 4 Argon (C++ black hole, realtime, told to make it scientifically accurate). 4 months ago this would be amazing but now this is horrible compared to even GLM 5.3 Flash on this test. I also kept hitting provider errors and the reasoning summaries are awful too "I've made a significant breakthrough regarding the Flamm funnel's orientation." like no you didn't shut up. Google is not back.