inference providers are creating the market that squeezes their own margins the basic economics of inference companies is selling tokens and paying for gpu time. what they earn from each gpu-hour depends on 3 things - the price of a token, # of tokens the gpu produces, and how much of the hour the gpu is used from one side, more providers are competing for the same gpus (pushing the cost of gpus up). from the other, they're all selling the same open models, which pushes the price of tokens down to get out of this problem, I think the only way is to get more tokens out of every gpu-hour. better software, or being differentiated in being able to run inference on cheaper hardware it's hard to tell which providers will survive because demand is growing fast enough so that everyone has customers. demand may not actually slow down, though I think supply will catch up

It seems the first market correction in AI is underway. Musk believes many neoclouds will run out of money. But there's also an advantage to this, because the acquired compute won't disappear; instead, it will go to companies that already have strong revenue streams. Therefore, no "bubble" will burst.

great post & chart. interesting... bit of narrative violation: >The inference business is more profitable than ever. >Compute cost per token = Gpu rental cost / billable tokens produced >Purely serving closed source tokens is becoming a less economically feasible business model. Why? Your input cost is rising while the price you charge customers is falling on a like for like basis. Anyone whose business is essentially reselling closed-source API tokens without proprietary efficiency advantages or other differentiation is getting margin-compressed.

As someone with a portfolio of energized land assets here is what I am seeing: 1. Neoclouds are disorganized and poorly capitalized. Some are definitely better than others however. 2. Hyperscalers are intractable, bureaucratic organizations that take 3 months to do an NDA and want 18mo escrow periods. We built our portfolio by competing against these terms because of our speed and simplicity. The more future assets orgs like ours win, the bigger the problem for the hyperscalers downstream (see 3). 3. There are delays in the existing pipeline that will catch people off guard in 2027. Hyperacalers will (obviously) react by focusing on their own internal compute needs over any third party. If the third party is a model company, they will tell them to go through their model router where they will also take a non-trivial vig. Them telling Ant they have no compute for them (rumored in Ant’s S-1) if true is a huge tell that this is already happening. All of this is to say that it does look like there will be a lot of partially stranded compute and compute sitting in inventory in 2027 than is currently forecasted. This is bought and paid for but effectively underearning (below total cost of capital) or collecting dust. We see this as true for everyone - model companies, hyperscalers, neoclouds except ONE (you can guess which one). This dislocation will benefit two kinds of orgs: the non-bureaucratic big scalers who can build industrial scale quickly and the nimble, well capitalized players feeding them. 2027 will be a turbulent year because it will clear out a bunch of wrong/bad bets. It will also be a consolidation year for a few.

Hard to see how hardware that fully pays for itself in less than 3 years and has significant residual value will result in low returns on capital.

I think we’re going to see a lot of stranded compute in partially complete data centers