@dylan522p

SemiAnalysis AI Infrastructure Research and Consulting DMs are open for consulting, quotes, or to talk shop, Opinions my own

Joined April 2018
FlashAttention was a mega kernel @marksaroufim @tri_dao @__tensorcore__ @vinodg – at San Jose, CA
17
5
1
137
19,298
I'm doing a new startup with @asciimike It's Chinese leg extension surgery Except for you break your skull and expand your brain – at San Francisco, CA
15
2
49
20,430
The two worst things for society Both called DCs Diet Coke Datacenters
79
4
6
290
68,096
Dylan Patel retweeted
A $200 AI subscription can be worth $12,000 in tokens. Which plan delivers it is not close. "They're selling to willing buyers at the fair market price. This is capitalism." "You can pay $200 for a Claude plan and get $12,000 of Opus 5.5 tokens at API pricing. So that's obviously a good deal." "You have to say the $200 plan, when running a particular workload on a particular model, gives you some dollar amount. But then you also have 5.5, which is a beast. $6K or $5,000 worth of value." "And when you do that comparison, it's not even a question of who's providing more value. ChatGPT versus Anthropic subscriptions." @maxkan and Andrew Megalaa join @JordanNanos this week
25
5
2
123
37,961
Mistral 4 is pretty good at lawyer bench and cyber security bench, but give it's European heritage, one can only assume it's also good at regulation bench
Meet Mistral Large 4, aka Le Chonk. • 1T parameters, natively multimodal. 49B active. It is the best open weights model from US or Europe on aggregated benchmarks. • State-of-the-art on critical workloads, including cyber defense, manufacturing and finance and it surpasses closed frontier models on visual grounding. • Forged in Europe end-to-end and is deployable from Europe via our own Mistral Cloud infrastructure. • Available to all via API today. Working with cybersecurity partners privately. Open weights release end of October.
35
10
2
294
46,518
"And antisemitism will be dead in about 6 months once Sydney Sweeney converts so don't worry bout it" - @Chaimeisenberg Incredible things are happening in SemiAnalysis Slack
26
4
2
232
43,582
The difference in what Anthropic and OpenAI as well as many others give you in value for a subscription is insane. We tested Meta, Minimax, Kimi, Grok, and Cursor subscriptions – at San Francisco, CA
It doesn't make sense to say a subscription plan is worth $ X in isolation. The way to think about subscriptions is that your monthly payment grants you some numbers of “credits”. Each (model, token type) combo consumes a different amount of credits. Because credit cost ratios can differ dramatically from API price ratios, the “value” of the same plan changes depending on what model and workload you’re running. Here’s how the API-equivalent value of the same $200/month Claude plan changes depending on the model and workload. (1/6)🧵
73
24
4
560
116,422
Long live the short king newsletter.semianalysis.com/… – at Sedona, AZ
Replying to @DirtyTesLa
We cut our RAM in half for the Tesla AI5 chip (now 72GB of LP5) and 1/3 for AI6 (now 144GB of LP6). This was the only way to get enough volume for Optimus production and greatly reduces cost. As it turns out, we think this will have a negligible effect on Optimus performance, as memory bandwidth is a bigger limiting factor than total memory storage (bandwidth was held constant).
14
6
2
318
103,610
A lot of people tell me they use agents to build xyz Thing is, some people have an accent or say it quickly And so I hear we use Asians to build xyz Which is like... Well ya that's been the global economy for the last few decades. – at Santa Barbara, CA
28
5
6
476
61,894
Can't invest in Anthropic at 2 trillion because it could be a 0 and I'm fucked, or it could be 20 trillion, but at 20 trillion we are all fucked.
193
259
62
7,772
378,128
Dylan Patel retweeted
AI is making papers cheaper to produce. ICLR submissions: 4,938 (2023), 7,262 (2024), 11,603 (2025), 19,525 (2026). Reported 2027 IDs exceed 62K, above roughly 56K paper submissions in all previous years COMBINED. Can reviewers keep up? (1/6)🧵
10
7
4
82
32,935
Rubin HBM Despec = Nvidia cutting coke – at Tamalpais Valley, CA
Fentanyl Grade Compute: Why a GB300 Rack Will Out-Price Blow by Weight in 2030 A GB300 NVL72 weighs roughly 1,580 kg fully populated and recent purchase orders put it at $5M per rack. That is $3,165/kg. Strip out the 1.5 tons of busbar, manifold and coolant and the GPU packages alone are well into gold territory, but we're pricing the rack, because that's what you actually take delivery of. Where that sits on the illicit commodity curve today ($/kg): $2,400 Cannabis flower $3,165 GB300 NVL72 $3,500 Fentanyl $28,000 Cocaine $65,000 Heroin $138,000 Gold So today NVIDIA ships a product that is denser in value than weed, roughly fentanyl-grade, and still an order of magnitude short of cocaine. For now... GB200 NVL72 was $4M, GB300 is ~$5M, and Vera Rubin NVL72 is already quoted at up to $8.8M at essentially the same rack mass. Rack weight is constrained by the datacenter floor loading; rack price is constrained by nothing. That's ~1.75x $/kg per generation with HBM, CoWoS and power all supply-limited through the decade. Cocaine: Colombia's new president has pledged a hard-line security crackdown with $1B in US aid behind it Every prior crackdown has consolidated the industry into fewer, better-capitalized operators with better logistics and higher yields per hectare. Consolidation is deflationary. We model US wholesale drifting from ~$28k/kg toward ~$12k/kg by 2030 as the supply chain professionalizes. **Crossover: 2030.** Our extrapolated NVL72-class rack hits ~$17k/kg while wholesale cocaine falls to ~$12k/kg. At that point the rational cartel pivots to smuggling racks, except a rack has a fixed 500+ kW power draw, a 3,300 lb forklift requirement and an export-control regime that is actually enforced. Cocaine has none of these problems, which is why it will remain the superior product for anyone without a substation. Key risks to the thesis: US retail coke is still ~$60–200/g, i.e. $60k–200k/kg so the crossover only holds at wholesale. Retail compute (H100 hour on a neocloud) is also marked up, so we consider this an apples-to-apples wholesale comparison. Full model available to Coke Research subscribers.
18
2
1
157
48,749
Dylan Patel retweeted
The Chinese AI Infrastructure Boom: Introducing the SemiAnalysis China Datacenter Model 1,000+ facilities across 60+ operators mapped, built retail-first and flipped by AI, largest hyperscaler leases 1/5 national capacity, 100MW in 12 months, Eastern Data Western Compute newsletter.semianalysis.com/…
28
65
18
408
123,039
Fentanyl Grade Compute: Why a GB300 Rack Will Out-Price Blow by Weight in 2030 A GB300 NVL72 weighs roughly 1,580 kg fully populated and recent purchase orders put it at $5M per rack. That is $3,165/kg. Strip out the 1.5 tons of busbar, manifold and coolant and the GPU packages alone are well into gold territory, but we're pricing the rack, because that's what you actually take delivery of. Where that sits on the illicit commodity curve today ($/kg): $2,400 Cannabis flower $3,165 GB300 NVL72 $3,500 Fentanyl $28,000 Cocaine $65,000 Heroin $138,000 Gold So today NVIDIA ships a product that is denser in value than weed, roughly fentanyl-grade, and still an order of magnitude short of cocaine. For now... GB200 NVL72 was $4M, GB300 is ~$5M, and Vera Rubin NVL72 is already quoted at up to $8.8M at essentially the same rack mass. Rack weight is constrained by the datacenter floor loading; rack price is constrained by nothing. That's ~1.75x $/kg per generation with HBM, CoWoS and power all supply-limited through the decade. Cocaine: Colombia's new president has pledged a hard-line security crackdown with $1B in US aid behind it Every prior crackdown has consolidated the industry into fewer, better-capitalized operators with better logistics and higher yields per hectare. Consolidation is deflationary. We model US wholesale drifting from ~$28k/kg toward ~$12k/kg by 2030 as the supply chain professionalizes. **Crossover: 2030.** Our extrapolated NVL72-class rack hits ~$17k/kg while wholesale cocaine falls to ~$12k/kg. At that point the rational cartel pivots to smuggling racks, except a rack has a fixed 500+ kW power draw, a 3,300 lb forklift requirement and an export-control regime that is actually enforced. Cocaine has none of these problems, which is why it will remain the superior product for anyone without a substation. Key risks to the thesis: US retail coke is still ~$60–200/g, i.e. $60k–200k/kg so the crossover only holds at wholesale. Retail compute (H100 hour on a neocloud) is also marked up, so we consider this an apples-to-apples wholesale comparison. Full model available to Coke Research subscribers. – at San Francisco, CA
73
79
46
923
420,781
Everytime AI spend grows where I flirt with token budgeting, Anthropic + OpenAI save me Astra had a spike then settled down lower Opus 5.5 it looks like spend is lowered for current usecases Of course people figure out new things to do Spend goes up again ROI keeps increasing – at San Francisco, CA
46
6
4
336
38,736
Dylan Patel retweeted
ALERT ALERT ALERT 🚨 🚨 🚨 VLLM MAINTAINERS HAVE JUST SHOWN THAT TPUv7 CAN GET 700 tok/s/user,  56% BETTER PERFORMANCE THAN NVIDIA GB200 NVL72 THROUGH MEGAKERNEL OPTIMIZATION ON KIMI K3. As we said awhile ago, the TPU externalization of software is full steam ahead. This is ultra important to follow the progress of this.
66
112
38
1,580
297,214
Spoke with AMD, they did not ship / sell this dual use chip to this firm and they're investigating the sourcing. This violates their policies + they work with authorities to take action on violations I am very concerned about the fact that pricing discrepancy is so large. I have heard from multiple folks that there are bitstream compatible FPGAs not from AMD that are floating around too. – at West Menlo Park, CA
US supply chains for electronics suck so much AMD found a business opportunity in dumping US military chips for 1/4 of the cost in China. US list price is $36k for this chip, with $4-5k to US companies at volume, but it's getting quoted $1k in China to crowdfunding campaigns AMD needs to be investigated for treason. it's not just export violations, but it's literally selling military end use components to an adversary for less than your home country.... Pharma companies are also similarly treasonous digikey.com/en/products/deta… crowdsupply.com/puzhi/pzsdr-…
24
8
6
201
78,722
I wanted to use this photo because it's the actual package and looks way cooler. The black area above is the DRAM but not milled yet. Cross section from our Hitachi XTEM coming soon 🤫
We took the silicon out of the silicon. Quick turn from SemiAnalysis STEEL Teardown Lab: iPhone 18 Pro Max, A20 silicon on TSMC N2. more to follow...
19
10
1
155
44,800
US supply chains for electronics suck so much AMD found a business opportunity in dumping US military chips for 1/4 of the cost in China. US list price is $36k for this chip, with $4-5k to US companies at volume, but it's getting quoted $1k in China to crowdfunding campaigns AMD needs to be investigated for treason. it's not just export violations, but it's literally selling military end use components to an adversary for less than your home country.... Pharma companies are also similarly treasonous digikey.com/en/products/deta… crowdsupply.com/puzhi/pzsdr-…
90
62
35
1,041
402,341