
SIGMOD Systems Award for Amazon Aurora · Creator of AWS SimSpace Weaver · Prev. Microsoft Research + Amazon Research · High-school dropout → MSCS + AI, @UW ·
us-west-2
Joined February 2007
- Tweets2.9K
- Following1K
- Followers4.2K
- Likes8K
Pinned Tweet
If AGI kills us all, it won't be the model's fault. It'll be the duct tape and footguns we wrap it in.
For those of you not yet familiar, this is what we in the biz like to refer to as an *aura* chart (hi Tibo).
As you can here in Figure 1, values appear to have flatlined after an extended period on life support, representing a complete and total collapse of aura.
Also, OpenAI's capacity issues appear to have improved.
1
3
212
I've seen too much of this nonsense on my timeline recently to not respond. Especially after watching people who don't know better get on the hype train and ponder investing in a company that every VC already turned down. So please allow me the opportunity to briefly provide some friendly advice and educate you on this topic before you light your money on fire:
Past symbolic failures were not from a lack of GPUs:
Expert systems, pure program synthesis, and GOFAI hit walls on perception, uncertainty, and scaling long before hardware was the constraint. Interaction nets' and optimal reduction's contributions include sharing computation and parallelism but do not automatically solve representation learning or statistical inference over messy data.
But this isn't the damning part. The nonsense claims about beating neural nets with some magical representation are.
The end representation of a function is what matters. And for the functions that actually get learned from large noisy data, a neural net is already a highly compressed, distributed encoding of that function.
Searching a symbolic program space for an observationally identical behavior can only succeed in a few ways:
Rediscovering, or approximating, a structure that is computationally equivalent to the net: i.e. layered compositions of simple continuous operations over high-dimensional vectors. Congratulations, you have reinvented a neural net in a different syntax, and any asymptotic gains have to come from the evaluation strategy not from escaping the representation.
Or you can expand the computation into an explicit unrolling: essentially enumerating or caching enough of the intermediate state space that the program becomes a gigantic lookup or decision structure. That trades the net’s compact parameter vector for space, and the space cost is usually much much worse once the function is rich enough.
There is no free third option in which a short discrete program magically computes the same high-dimensional statistical mapping more efficiently. The “closed set of functions + data” case collapses back into the first case.
Architecture tweaks inside the neural-net family stay inside that family and can improve constants or inductive bias. But they do not open a new representation class that is fundamentally more efficient for the same class of functions.
Differentiability is the reason we can even find these compact representations at all. Discrete search over program space does not automatically locate the same low-description-length solutions and the optimization landscape is far harder. And once a function has been found, its evaluation cost is determined by how it is represented. The claim that a pure symbolic search will routinely beat an already-trained, heavily optimized net on that cost is just a baseless assertion with no information-theoretic grounding.
So the whole premise reduces to “maybe there exist functions whose shortest efficient programs are not neural-net-like.” For next-token prediction on natural language, or similar perceptual/statistical tasks, that has not ever been shown to be the case, and the size of the parameter vectors relative to the data already indicates those functions are not especially compressible in conventional symbolic languages.
Ergo:
1/ Do not give this project your money. It is a terrible investment.
2/ This premise for replacing NN and creating AGI is not going anywhere and the claims are pseudo-scientific nonsense.
3/ The framework itself can still be cool.
*This is investment advice.
Replying to @VictorTaelin
Since our pitch deck was uninformative, let me try to explain our whole thesis. Sorry if any part of it sounds obvious.
I argue that any reasonable definition of AGI must imply a system capable of learning as efficiently as a human. LLMs obviously don't pass this bar. When you talk to them, they don't learn anything, since the weights don't change. So, unless true infinite context lands, or GPUs become 10000x cheaper, the current methods won't scale to AGI, at least in that sense.
(Note that this has nothing to do with raw capabilities. Perhaps Fable 10 will be building Dyson spheres, yet still be unable to internalize new knowledge as efficiently as humans do. So, one might ask if we even *need* AGI in the sense above. I think we do, if only because a drastic drop in pre-training costs would make AI more democratic, and prevent one of the worst outcomes in my mind: over-concentration of power.)
As you know, modern AI is based on neural networks. What not everyone knows is that NNs are just very, very slow programs. Nobody in their right mind would sort a list using NNs, because that'd be millions of times slower than just using C. So, why do we implement *intelligence* with NNs?
As a thought experiment, imagine if we could take Fable's weights and distill them, not into a smaller model, but into a raw C program that was observationally identical to it. Literally, a C file with astronomically complex, tangled procedures that, when compiled and executed, behaved just like Fable 5.2.
If such a thing were possible, then we could have Fable at a million tokens per second. And here's the cool part: such a thing *is* possible. At least, as far as computer science is concerned, nothing says it isn't. So, why don't we do just that?
Because we don't know how to *train* a raw program.
Neural networks are slow, but they're differentiable, so we know how to train them. Step by step, slowly, but surely. Raw programs are fast, but we have no methods capable of training them. So we don't. We just stick to NNs, and accept that it takes $50 billion to train a model. We take that as "normal". But is it?
In theory, it shouldn't be like that. As the old saying goes: first you make it work, then you make it fast. We know AI works. Now, we will make it fast. That's the premise behind Symbolic AI. HOC's first milestone is to train a model comparable to GPT-2, with no NNs. If that works, we might have the fastest inference engine in the world. Not by 2x, but by 1000x.
So, in a way, HOC is like DeepSeek or Jev, in the sense that we're all about efficiency, but in a much more extreme way. People are aiming for hardware, but we believe there are still 1000x gains to extract in the software layers, if we're willing to rethink the whole stack from literal scratch.
Many laugh and say this is impossible, but they say so with no basis. They just observed that many failed to build symbolic AI models in the past, and concluded it is impossible, or very hard. That's not a formal proof. In fact, there are many reasons people in the past could've failed, yet the actual problem may end up being easier than most anticipate. Symbolic AI flourished half a *century* ago. Back then, they didn't have:
- As much knowledge as we do
- Massive GPU clusters
- Methods to scale Symbolic AI (Bend3, HVM5!)
So, just like everything that once seemed impossible, there's one core prerequisite to making it possible: *trying*.
That's what we'll do!
I don't think I'm smarter than all the brilliant researchers who worked on this problem in the past, and failed. I just think I'm equipped with tools, compute, and knowledge that they didn't have. So, standing on the shoulders of giants, and with enough dedication, I don't think it is so unlikely we stumble upon some symbolic algorithm from which, given enough compute, unexpected capabilities emerge.
If it does work, I do think business is straightforward. At the very least, we would be able to sell incredibly high quality datasets to big labs, but, at that point, just selling inference might work as well. Business is obviously not the wall, it is all on whether such massive software-based speedups exist, and whether we're able to discover them.
1
9
388
OpenAI is several thousand employees deep and is still trying to assemble a team.
You’re NGMI. Just put the reset in the bag, lil bro.
Replying to @embw_l0x
We're assembling one of the best teams ever and I do talk to folks at Anthropic every once in a while. Couldn't be more excited for the months ahead.
5
540
And now you have the explanation for why Opus 5.5 was just kneecapped in the last week.
Introducing Claude Fable 5.2 and Claude Mythos 5.2, designed to tackle the most daunting technical challenges.
They’re capable collaborators on ambitious projects in many fields, and show improvements in areas like coding, reasoning, and vision.
4
378
AI doomer timeline palette cleanser.
This has to be some of the sickest shit I’ve seen in a long time.
4
247
We've been through this before, friends.
I really hope people stop clamoring over the latest frontier model release long enough to realize that every frontier model is mere weeks away from having the inference compute rugged from underneath it.
Compute is constantly being reallocated and margins are being preserved at the expense of your ability to produce useful work from these models. The benchmarks are all very clear: a model that is not allowed to reason will confidently generate useless garbage. You cannot overcome this with more guidelines and skills. You cannot steer a model that does not think to success.
This is an impressively misguided article. But it is also curiously worth reading at the same time.
They seem to contort every leap of logic to justify an insane reliance upon a single model from a single provider. Their AI spend is insane and their infrastructure spend is also unnecessarily wasteful.
For a company that is so AI-pilled, it's ironic that they can't see the impact of their single-model dependency on Fable. They claim only Fable can save them: but for an org so dependent on Fable they also can't seem to articulate the impact of Anthropic's documented shenanigans with delivering stable inference compute.
Still, there are real hints of brilliance in the thinking. And some of the shapes do feel directionally correct. These are clearly smart people that just appear to be lost in the sauce and possibly caught up in the vapor of AI flatulence.
Take some time to read the article. They have put a lot of someone's money on the line, even if it's not theirs, and are clearly willing to put someone else's money where their mouths are. And who knows, maybe their bet will pay off.
4
330
Everyone thinks the cyber sky is falling but we are actually headed towards a new digital libertarian utopia.
All the bloatware will be gone. All the adware will be gone. All the telemetry will be gone. Everything will be customized to your liking. And everything you use will be secure.
You will finally get to own the things you buy instead of the things you buy owning you.
A lot of people seem confused about the availability of larger context window sizes in Codex.
Please allow me to clear it up for you (with pictures).
Let me also clear up one other point of confusion: this has been available to you since early August.
Day 7/
We have increased default context length for GPT-6.1 Sol to 500k when used through Codex & Work, up from 272k or an ~ 80% increase.
We will go higher in the future, but it helps reduce occurrence of compactions and with tasks that require keeping a lot of context and nuance over periods of time. It was carefully tested and is the new sweet spot for most of our users. You can still override it if you want to be higher our lower, with maximum context length of GPT-6.1 Sol of 1M.
Folks, the entire 1.05m context window is already acceptable free of charge with your Codex subscription.
Stop allowing these companies to gate-keep shit that is already free.
Day 7/
We have increased default context length for GPT-6.1 Sol to 500k when used through Codex & Work, up from 272k or an ~ 80% increase.
We will go higher in the future, but it helps reduce occurrence of compactions and with tasks that require keeping a lot of context and nuance over periods of time. It was carefully tested and is the new sweet spot for most of our users. You can still override it if you want to be higher our lower, with maximum context length of GPT-6.1 Sol of 1M.
I regret letting Codex ever run with a /goal prompt or launch sub-agents.
It doesn't matter what model it is at this point. The Sol and Astra lineage are completely RL fried.
It doesn't matter how tight the prompt is or the agent instructions are. I've even had prompts and instructions externally reviewed and rewritten, but it doesn't change the behavior.
The model always goes off track, ignores scope, and finds a way to digress into scaffolding, certification machinery, evidence gathering or audits. Even with the literal words barring it from doing so are right there in the prompt, plan, or instruction file.
And it always ends in an absolute mess of junk files, slop code, and an account drained of limits.
Codex can only be used effectively through the tightest of handholding. Keep one-shotting whatever you want if it's working for you. But real work on an existing system is excruciating and nearly impossible.
They are shipping models they know don't follow directions and are nearly useless for real long-horizon work.
You have to be a real jerk to admit you've been operating subscription accounts at half speed and that all you had to do was flip a switch to fix it.
Day 1/
We have optimized the default speed to be ~50% faster across GPT-6 Astra and GPT-6.1 Sol through the subscription across all our products and partners using Sign in With ChatGPT (including OpenCode, Pi, Amp, Devin, ...).
No changes needed on your end and this should be felt within the next two hours.
Just in case the games aren't clear at this point:
teapawt.org/BrenBuilds/status/2107…
If you let these jamokes get away with this crap they will perpetually route you to the dumbest model they possibly can, using the tiniest reasoning budget they can plausibly allocate you, all while constantly compacting your context window usage.
The model will be slow, stupid, forgetful, and untrustworthy.
But, on the bright side, there probably won't be any usage limits to worry about.
Tibo confirmed that OpenAI plans to merge Codex and ChatGPT capabilities, with the goal of making the experience less complex for users:
“…and for Codex and ChatGPT, we have Chat and the Work toggle, and we’re going to merge them. We’ve heard the feedback loud and clear: people love the capabilities in Work, but sometimes you want to use Chat because it’s a little faster and more pleasant to use. So we’re merging the two and reducing that complexity.”



















