Gemini Researcher. Formerly: Founder/CEO MutableAI (acq by Google). Theoretical Particle Physics. Views my own.
Bay Area
Joined December 2021
- Tweets1.5K
- Following750
- Followers1.6K
- Likes3.1K
Omar Shams retweeted
Google’s Gemini 4 Carbon is the new #1 on the Artificial Analysis Intelligence Index v5, and second only to Claude Opus 5.5 on the Artificial Analysis Coding Agent Index
Gemini 4 Carbon scores 59 on Intelligence Index v5, driven by leading performance in complex reasoning, knowledge, and scientific tasks. It also has large gains in coding and knowledge work, particularly when compared to other Gemini models, but it is not leading in all these areas.
As with launch of Gemini 4 Argon, Gemini 4 Carbon is being rolled out to selected users and is not yet publicly available. Google is offering a 50% launch discount, with no confirmed end date. All our charts and figures use standard pricing for both Gemini 4 Carbon and Argon.
Key results for Gemini 4 Carbon with high reasoning:
➤ Gemini 4 Carbon leads the Intelligence Index v5, with the #1 score in 4 of the 9 Index evaluations: HLE-Diamond, Terminal-Bench-Science, AA-Omniscience, AutomationBench-AA. Carbon makes substantial gains elsewhere but is not leading in coding and knowledge work
➤ Behind only Claude Opus 5.5 on coding agents: With Antigravity CLI, Gemini 4 Carbon (high) scores 59 only 3 points behind Claude Code with Claude Opus 5.5 (max, 62) on the Artificial Analysis Coding Agent Index. Carbon ranks second on both Terminal-Bench 4.0 (64%) and FrontierCode v1.1 (51%), behind Claude Opus 5.5 (max) at 56%. Google has historically underperformed in this area and so this improvement stands out
➤ Improved agentic knowledge work, but not yet leading: On AA-Briefcase, Gemini 4 Carbon (high) scores 1609 Elo, a 126-point improvement over Gemini 4 Argon (high, 1483), but still behind models including Grok 4.7 (xhigh, 1638), Muse Wonder (max, 1659) and Kimi K3.1 Pro (1678). The largest gain comes from Presentation, up 293 Elo to 1598, alongside a 120-point improvement in Analytical Quality to 1671
➤ Lower Cost per Task than Claude Opus 5.5 for comparable overall intelligence: Gemini 4 Carbon costs $7.76 per Intelligence Index task, ~23% less than Claude Opus 5.5 (max, $10.04) but ~1.2x GPT-6 Astra (max, $6.25). It uses 123k output tokens per task, 29% fewer than Claude Opus 5.5 (max, 173k). At the same $4/$20 standard pricing, its Cost per Task is in line with Gemini 4 Argon ($7.68), while scoring 9 points higher on the Intelligence Index
➤ Leading AA-Omniscience v1.1 score, with low hallucination: Gemini 4 Carbon scores 54 on our recently updated AA-Omniscience v1.1, 6 points ahead of Claude Opus 5.5 (max, 48) and 8 points ahead of Gemini 4 Argon (46). Accuracy (% correct) rises 10 points over Argon to 58%, while its hallucination rate rises from 4% to 11%. This is still the second lowest of any model, behind only Argon, compared with 50% for both Claude Opus 5.5 (max) and GPT-6 Astra (max)
Key model details:
➤ Context Window: 1M tokens
➤ Multimodality: Text, image, video, and speech input, with text output
➤ Pricing: $4/$20 per 1M input/output tokens, the same as Gemini 4 Argon’s standard pricing, with a 50% discount. Cached input tokens receive a 95% discount ($0.20 per 1M)
Congratulations to the @GoogleDeepMind team!
Omar Shams retweeted
I did say we'd be releasing our models periodically...introducing Gemini 4 Carbon, our next frontier model.
Following Argon's release just two weeks ago, Carbon shows the pace at which we’re pushing the frontier, it ranks #1 on the recently updated v5.0 of @ArtificialAnlys Intelligence Index at 59. It also leads across key cybersecurity evaluations scoring 88.2% on SEC-bench Pro for finding vulnerabilities, 54.8% on ExploitGym for exploitation, and 71.0% on CWE-bench for patching.
We’re seeing significant improvements in long-horizon tasks even compared to Argon, and we’ve strengthened frontier safeguards.
As we did with Argon, we’re going to roll it out gradually + responsibly. Some trusted cyber defenders will get access starting this week. We'll be expanding it to more partners through our Fairwind Program, ahead of wider availability soon, so hang tight!
Omar Shams retweeted
people who understood the NS solution told me that it contains 10 field-shaping breakthroughs as part of the solution. The idea that the models only produce results in the convex hull of human knowledge (whatever that means) is misguided.
Omar Shams retweeted
Introducing Gemini 4 Argon – our new frontier model.
It’s built for complex workflows across coding, enterprise knowledge work, and cybersecurity defense – rolling out today to a set of trusted testers through our Fairwind Program.
Wittgenstein more relevant than ever. Unironically, the limits of language are soon going to be the limits of our world.
Problems one of the best theoretical physicists around expects we crack soon with AI
Even doing three loops is a PITA and was something you have to check painstakingly to make sure you’re not messing something up. Nine loops is an amazing result.
New on the Science Blog: Yes, Claude can do Nine Loops.
Theoretical physicists predict how particles behave using formulas called scattering amplitudes. These are notoriously hard to compute, so researchers work with layers of increasingly fine corrections called “loops”—each added loop makes the answer more precise but takes exponentially more computation. Most calculations stop at two or three loops. Eight loops was the previous record in a simplified model physicists use as a testing ground (planar N=4 super-Yang-Mills), set by SLAC's Lance Dixon and collaborators.
Last month, physicist and science writer @4gravitons issued a challenge: could an AI push past eight loops in this model, using only the compute budget an academic could reasonably access?
Given a single prompt describing the nine-loop problem, Claude ran largely unsupervised for days in Claude Science and solved it using methods developed by Dixon and his colleagues, at a total cost of a few thousand dollars. Dixon independently verified the result, and von Hippel wrote about the experience for our blog.
Read more: anthropic.com/research/yes-c…
In the long run the universe turns to iron* and before that all codebases converge to Rust
*unless protons can decay
Quantum gravity is definitely getting solved in the next ten years. Putting my chips on a non commutative algebra that can pixelate space time and is identical to a string theory in a limit.
The fact that AGI is an inevitable consequence of industrial processes feels like an update and improvement on Democritus (atomic hypothesis).
In the Feynman Lectures on Physics:
“If, in some cataclysm, all of scientific knowledge were to be destroyed, and only one sentence passed on to the next generation of creatures, what statement would contain the most information in the fewest words? I believe it is the atomic hypothesis (or the atomic fact, or whatever you want to call it) that all things are made of atoms—little particles that move around in perpetual motion, attracting each other when they are a little distance apart, but repelling upon being squeezed into one another. In that one sentence, you will see, there is an enormous amount of information about the world, if just a little imagination and thinking are applied.”
If we kept Feynman’s original proclamation we’d be tempted to improve it by saying all knowledge can be simulated by large matrices learning to reproduce corpuses of human text by trial and error through gradient descent.
One of the most tragic things to see in Silicon Valley is someone who used to be extremely ambitious achieve a modicum of success and then promptly lose all their ambition.
Nice!
Codex and I spent several days testing this model back and forth. We were both very impressed.
Anyway, I shared the launch with codex afterwards and they were so excited our work made it in.
Check it out
blog.google/innovation-and-a…
During one pitch for my last co (AI for code) the partner said there’s no hope for any startup here because Github had “already won the space” 🫠
Another told me AI for code is “impossible” 😂
Being a smol time angel investor has given me a little more sympathy but not much
I was once pitching in a board room at a top 3 VC firm for a $15M Series A.
12 people in the meeting. One of the GPs fully fell asleep. Out cold for 30+ minutes. Nobody acknowledged it. Everyone just kept going.
I kept presenting my Series A slides to an unconscious man in a Herman Miller chair and somehow that was considered normal. That's venture capital.
You might fly across the country to perform for people who may or may not be conscious.
It's a dance.
And sometimes you lead and sometimes you follow and sometimes your partner is unconscious.
If you're raising right now, just know: every founder has a story like this. The process is weird. The power dynamic is weird. You're not crazy for thinking it's weird.
No one talks about it because they want to continue raising. But I'm happy to stick my neck out there.
It is weird.
i used to believe progress in robotics was bottlenecked by actuators etc now i realize i was mistaken and it's more limited by compute
Atomic Semi moving to Austin is kind of a big deal.
I’m thankful for 4 years in California .
But, the Next Phase is in Texas.
We shift our hiring focus to build huge operations in Austin, Texas.
Join us ! The team is really great. @fab2
New 120,000 square feet HQ for R&D and production:
