Can your GPU run that model? Find out in 10 seconds. Free VRAM calculator + no-hype hardware guides for local AI.
Joined June 2026
- Tweets476
- Following139
- Followers248
- Likes495
How much power does an RTX 5090 need for local AI?
One run, Qwen 3.8 Flash Next NVFP4 on the Basalt engine, power limit stepped from 400 W to 600 W:
400 W: 115.8 tok/s
475 W: 132.1 tok/s
575 W (stock): 135.1 tok/s
600 W: 138.5 tok/s
At 475 W the card keeps 98% of its stock speed on 100 W less.
Below 475 W the speed falls fast. Above it, each extra watt buys very little.
A 763B model with every expert held in memory, on a mini PC.
DeepSeek V4.1 Flash 1017 on AMD's Ryzen AI Max+ PRO 495 with 192 GB: 43.2 tok/s on average over three answers, 256K context, all 15,744 experts resident, 34 GiB still free.
It has the same 273 GB/s memory bandwidth as a DGX Spark. The difference is size: 160 GB given to the GPU, against 112 to 119 GiB usable on a Spark. This build needs 114 GiB.
A community quant with refusals removed, not the stock model, and one user's machine.
vramcalculator.com/gorgon-ha…
Updated Strata and your 12 GB NVIDIA card got slower? Part of that is on purpose.
Strata 0.2.7 added a speed-up for NVIDIA cards with 12 GB or less: answers up to 37% faster on an RTX 3060.
Strata 0.2.8 switched it off again by default. Users and the developer tied it to repeated text in long sessions.
To turn it back on: Models, Native engine settings, Speed, "Faster answers on 12 GB NVIDIA cards". Or launch with --set STRATA_ROUTE_PRIOR=1.5
The cost in Strata's own test: 4.5 of 10 agent tasks passed with it on, 5.6 with it off, inside their run-to-run spread.
It is not the whole story. A 24 GB RTX 3090 went from about 80 to 60 tok/s on the newest engine, with fewer experts kept on the card (13.0 GB against 15.8 GB). That report is still open.
NVIDIA's CMP 100-100 mining card ships with 6 of its 12 GB of HBM2 switched off.
@Roberty35570209 turned the rest back on. He put the Linux method on GitHub on 1 November, and on the 2nd a two-card test took both cards from 6 GB to 12 GB.
His numbers: two of them run Qwen 3.8 27B at 32 tok/s (8K context, MTP drafts). He paid about $25 for his, second-hand.
The catch: the PCIe link stays at Gen1 x1, so a model takes 2 min 13 s to load.
The bigger CMP unlock, the 170HX from 8 to 64 GB, is the cover story of our guide: vramcalculator.com/sizing-gu…
The repo: github.com/6Robert123/cmp100…
It runs as root and ships a small helper binary whose exact source he's still tracking down, and the PCIe link is an open problem he asks for help with. Read it before you try it. We have not tested it.
Old DDR3 Xeons are supposed to be e-waste for local AI.
75 tok/s. Qwen 3.8 Flash Next, a 180B model, answering after a 172K-token prompt.
The box: two Xeon E5-2670 v2 with no AVX2, 64 GB of DDR3, PCIe 2.0. And 2x CMP 50HX mining cards, memory-modded from 10 GB to 20 GB each.
On 1 October Strata refused any CPU without AVX2. On 4 October it ran them.
xrip measured this box on his own build of Strata 0.2.0: 65.5 tok/s by default, 75.1 with 19 CPU workers, six runs.
Why it works: 98% of the experts sit on the 2x mining cards. The old CPU computes the other 2%.
Which old Xeons run it, and the October fight that got Strata there: Issue 2 of The Local LLM Sizing Guide, out tomorrow.
vramcalculator.com/sizing-gu…
Still holding mining cards? They have a second job now.
Issue 2 of our sizing guide shows, step by step, how the CMP 170HX goes from 8 GB to 64 GB. It also sizes GPUs and unified memory machines, including the new Macs, old DDR3 workstations and more.
Pay in crypto: $27 for October, November and December, with September included. Four issues.
vramcalculator.com/sizing-gu…
Apple's 512 GB Mac Studio is orderable now: $22,299.
The 256 GB version of the same M5 Ultra chip is $10,799. So what does the second 256 GB buy?
At 256K of context, by our calculator, mostly better copies of the same models:
MiniMax M3.1 Flash: Q2_K on 256 GB, Q8_0 on 512
GLM 5.3 Flash: Q3_K_M, then Q8_0
Ornith 1.5 397B: Q3_K_M, then Q6_K
Qwen 3.8 Flash Next: Q5_K_M, then Q8_0
And the 550B to 1T class moves in: DeepSeek V4.1 Flash 1017, GLM 5.3, Kimi K2.6 and Inkling 975B, whole, in memory, at 256K. 73 models fit against 58.
One thing to know before you pay: Strata's own Mac plan runs DeepSeek 1017 at 2 bits with every expert in memory on the 256 GB model too. It reads the lookup tables from the SSD.
Speed on this chip: DeepSeek 1017 at 54.5 tok/s (256 GB M5 Ultra, ds4, tested to 128K, @niklaslenz_ai). The weak side is prompts: 1,475 tok/s reading a 32K one.
We sized this Mac in September, before it shipped. The second image is a page of Issue 1: every model we size, on eight machines. Issue 2, out 31 October, has the full launch dossier, and Issue 1 comes with it.
Long chats and code, or long prompts all day: which is your workload?
vramcalculator.com/sizing-gu…
Project Maya calculator 0.7 alpha is out. This one is about the builds you actually download.
Check your machine before you commit 200+ GB. The abliterated DeepSeek Mix-128G and Mix-256G, plus the CYBER-FROST Flash Next, are in the picker and sized against your card, RAM and disk, with the per-GPU split the tool cannot see marked HOLD. So is every GLM, MiniMax and Qwen build.
The calculator engine got a real upgrade, and it is experimental. Support, pictures, drafters and the card limit now come from one set of rules, so the picker and the verdict cannot contradict each other. Cards allowed: 16, double the old 8. Fewer contradictions, which is the point of a sizing tool.
Accuracy Boost: the GLM baseline was re-run, 18,720 plans, none moved. 271 parity checks. 108 to 117 cards per family, re-read from Maya 1.1.34 on 28 October.
The UI additions are experimental too. Copy link puts your card, RAM, context and build in the URL: send it to a friend and they open the same machine and the same verdict. Copy summary writes the setup and the numbers as text, for a reply, an issue or your build notes. Rows and tabs work from the keyboard.
vramcalculator.com/project-m…
The MiniMax builds in that table are @PeasantSmith's, and Project Maya has been shipping model after model this month: MiniMax M3.1 Flash in three sizes, Qwen 3.8 Flash Next with the CYBER-FROST GGUF, DeepSeek V4.1 Flash 1017
github.com/mw00/project-maya
Nanbeige 4.3 33B fits a 24 GB card at Q4. That is the trap.
The Q4_K_M file is 20.79 GB. With the default cache, an RTX 3090 or 4090 gives you 16K of context. The model advertises 512K.
It is dense, with full attention on all 52 layers, so every token of context costs about 0.2 MB of VRAM. 256K of cache alone is 56 GB, more than two 3090s hold.
What actually works on one card, with the desktop running:
RTX 3090 / 4090, Q4_K_M, 4-bit cache: 32K
RTX 3090 / 4090, Q3_K_M, 8-bit cache: 64K
RTX 5090, Q4_K_M, 4-bit cache: 128K
16 GB cards: Q2_K only, 32K with a 4-bit cache
A 4-bit cache costs some quality. A smaller file costs more. Pick which one your work can lose.
So who is the 512K window for, exactly?
Your card, your context: vramcalculator.com
VRAMCalculator retweeted
Project Maya v1.1.29 is here, and it's a big one: MiniMax-M3.1-Flash now runs on Maya.
That makes three frontier open models you can run at home on your own GPUs: GLM-5.3-Flash, DeepSeek-V4.1-Flash and now MiniMax-M3.1-Flash, a 428B model with 23B active per token and up to a 1M context.
MiniMax gets its own engine, built by @needmorevram. We brought it up to the same standard as GLM and DeepSeek before shipping it:
- Runs on NVIDIA from V100 up, on Linux and Windows
- Fits itself to your PC: the setup picks the context your memory can hold, and the engine plans VRAM and RAM instead of failing
- A compact cache by default, so long contexts take half the memory with no loss in quality
- Pictures, tool calls and every one of MiniMax's thinking levels, with its own recommended settings out of the box
- The vision tower stays off your GPU until you send a picture, so the VRAM goes to the model
Huge thanks to @needmorevram, and to everyone who tests, reports and contributes to Maya. This is what community open source looks like.
Next up (WiP): our own Maya GGUF quants of MiniMax-M3.1-Flash, small enough for a 128 GB PC.
Link in the reply 👇
64 GB of VRAM is supposed to cost workstation money.
$1,300 to $1,700. Used. NVIDIA's CMP 170HX, a 2021 mining card.
Same GA100 chip as the A100, HBM2e at about 1.5 TB/s, faster than an RTX 4090. NVIDIA shipped it with 8 GB usable and left the other 56 GB locked on the board. A software unlock, cmpunlocker, opens all of it.
What one unlocked card holds, by our calculator, every layer on the card:
Qwen 3.8 Flash Next 180B, Q3_K_M, 64K context
Ornith 1.5 122B, Q3_K_M, 262K
Qwen 3.8 27B, Q8_0, 512K
Nanbeige 4.3 33B, Q8_0, 128K
LTT Labs measured Qwen 3.8 27B Q8_0 at 33 tok/s on one unlocked card. Two RTX 3090 Ti: 31.
A used A100 40 GB still sells for about $3,500.
The catch is a lottery. The 8 GB card opens to 64 GB. The 10 GB card stops at a stable 40. How to tell them apart before you pay, and which unlock to run on which, is the variant unlock sheet in Issue 2 of The Local LLM Sizing Guide, out 31 October.
So which one is in that listing?
vramcalculator.com/sizing-gu…
Before we loaded Ornith 1.5 122B, our calculator said it would use 77.7 GB at its full 262K context.
On an NVIDIA RTX PRO 6000 Blackwell Server Edition it used 76.4 GB. 1.3 GB on the safe side.
The card ran in a server with no screen. On your desktop the calculator also keeps 5% of the card free for your monitor and Windows, so a "fits" answer holds there too.
The run: whole model on the card, Ornith's own Q4_K_M, llama.cpp.
Answers at 88 tok/s with a 240K-token prompt loaded, 108 tok/s at 128K.
Every run we make, misses included: vramcalculator.com/vram-calc…
The Strata calculator 0.8 beta is live, and it now sizes DeepSeek V4.1 Flash 1017.
Strata 0.2.4 runs this 763B model on one 12 GB graphics card. The busiest experts sit in your RAM, the rest stream from your SSD.
Pick your card and RAM to see which of the three sets runs, how much disk it needs, and how much of it ends up on the SSD.
What to expect, by Strata's own figures: about 8 to 10 tok/s on an RTX 3060 12 GB with 46 GB of RAM. RAM matters most: @Tono_Ken3 got 38 tok/s with 256 GB.
vramcalculator.com/strata-ca…
ALT Strata calculator 0.8 beta card: a tilted screenshot of the calculator showing DeepSeek V4.1 Flash 1017 on an RTX 5070 12 GB with 64 GB of RAM, beside the new features: three ISTA-DASLab sets, Strata's own placement, 12 GB card, 46 to 64 GB of RAM, 239 to 274 GB of disk, real runs, Project Maya one click away.
New cards since the last update:
• Tesla P100 12GB. Pascal is experimental in Strata: its CUDA 12 package ran an IQ2_XS model on a P100. @VectorCrossProd
• NVIDIA's CMP mining cards: CMP 30HX 6GB, 40HX 8GB, 50HX 10GB, 90HX 10GB and 170HX 8GB.
• The CMP 170HX with its memory unlocked by cmpunlocker: 64 GB from the 8 GB card, 40 GB from the 10 GB card. Experimental, Linux only, and models load slowly over its PCIe link.
The engine and the files: Strata 0.2.4 by @coldniko (Terminal only for now, the app follows) and ISTA-DASLab's GSQ-RCO quants by @mhelcig.
The calculator places the model with Strata's own estimate, ported and checked against the original on 17,552 cases.
github.com/Niko1221/Strata/r…
