i should have posted this a long time ago hey, i'm sudo, and if you own a gpu and want to get your hands dirty with local ai, follow along i run hands on experiments with local models on every tier of hardware, 8gb, 12gb, 16gb, 24gb, 32gb, 64gb, 128gb, all the way up to 256gb and climbing, and i've been in the local ai space for a while now i also drop recommendations for the uncensored models and the weird ones, odd models and odd setups, so you can experiment and actually understand what's happening on your own hardware my goal is to get as many normal people as possible started in local ai, and give them a taste of what owning your thinking feels like on your own metal so if this moves something in you, listen to it, tell me in the comments what you want to see and i'll do my best to get you the answer, and if i can't, someone else will, the people in local ai are the most helpful crowd i've ever seen

if you are not harness engineering you probably not gonna make it anon

someone smarter than me explain this. why do we need a whole new GPU generation just to get more VRAM? can't manufacturers use the same chip and put more memory next to it?

which friend would build an entire startup just to avoid paying $20/month?

i'm falling for my own agent harness xpress agent is the harness i built for myself, it's rust and i can feel it in the speed, and offrail 27b is the model behind it, running on my laptop right now it pulls images off the web and lay them out right under its answer, i slid through them and opened the one i wanted, no more clicking through folders to find what it downloaded, and this one feature saved me so much time the model does the thinking, the harness decides what it sees and what it remembers, and you spend your whole day inside the harness, so it should fit the way you work > one binary under 10 mb > 14 mb of ram for the whole loop > finds my local model by itself > self learning loop it writes its own skills as it works, and on that same run it rewrote its image skill without me asking i think eventually you all will have to build your own agent harness the way you like it and the future of agents is rust

i might be stupid but if my SSD can store 4TB of data, why can't i use part of it as extra RAM when i run out? i know it's slower, but what exactly stops us from making this seamless?

here are the 1017 weights and the recipe, the launch build stays up next to them > weights: huggingface.co/sudoingx/Deep… > recipe: github.com/sudoingX/dgx-spar… 196 gb of weights plus deepseek's own 1017 engram shards, the launch ones differ, and the heavier dense and head runs sit in the repo's benchmarks with their speed

i rebuilt my deepseek v4.1 flash weights on the 1017 update, v1.2 is live on 2x dgx sparks 🐋 > structured output: 141.3 tok/s > code: 100.7 tok/s > prose: 59.2 tok/s and the quality held, my 2.70 bit weights scored against native 1017 on my own kl panel, the one other 1017 packs now report on > kld 0.0243 from native 1017 > 20% closer than libertaidai's 2.9 bit pack > top-1 agreement 95.47% > humaneval+, mbpp+, ifeval within noise > ifeval replies: theirs wins, 0.1135 vs 0.1199 deepseek shipped 1017 as a minor update, so i diffed it against the checkpoint my build came from, the config, tokenizer and tensor layout match byte for byte and every shard's content is new, in the slices i sampled about 27% of the 4 bit expert codes flipped and the built-in drafter moved the most there was nothing to patch, so i pulled the 510 gb checkpoint again, measured all 15,360 experts, ran a 16 hour encode with one spark down for almost 3 hours while the other carried on solo, then dense layers, head, drafter and every eval, about 25 hours from download to serving it runs about as fast as my launch build, which did 103.0 tok/s on code and 58.0 on prose, a verify round on code still takes 42.8 ms on both, the new weights just let the drafter land 4.31 tokens a round on that reply instead of 4.41 by the time mine finished, henryous and 0xsero had shipped 1017 packs built on my recipe, and henryous reports 0.0201 at 2.75 bits with heavier dense layers and head, closer to native than mine so i tried that mix on my own experts, dense at 5 bits and the head at 6 got me to 0.0219 for about 5% of my speed, i'm shipping the fast build and the heavier numbers are in the repo same engine as v1.1, so you keep the image and swap in the 1017 weights plus deepseek's 1017 engram shards next i'm training my own drafter for it to push more

30 years ago you upgraded vram with your thumb, then they took it away like the headphone jack

i asked one DUMB question and my replies turned into a hardware lab almost a million views and the thread went deep on vram mods and hbm, so if i got this right > gddr is soldered next to the die > 28 gbps a pin, a socket kills that > every chip owns 32 bits of the bus > 384 bit = 12 chips, clamshell = 24 > 48gb 4090 = clamshell + bios mod > hbm is on the package, no upgrades and then there's the rtx pro 6000, the same gb202 die as the 5090 with 96gb instead of 32 and a workstation price tag if you've modded vram onto a card yourself, drop the board below, i want to see every one

if you have never seen 4x dgx sparks dancing, here they are serving ling 3.1 flash for over 12 hours > about 20 w of gpu power per box > 49 to 55 °C after 12 hours serving > no restarts across a 12 hour build the spark's design is so power efficient for the work it does, and the heat flow and the scaling are just as well designed, nvidia supports clusters officially now i think in 2026 it's the best entry point i know into a desktop ai supercomputer that handles almost every ai workload, and the ecosystem is so much broader than 6 months ago it's basically a subscription canceling machine, from hosting your ci runners to serving your own models, and if you're planning to buy one don't overthink it, it's the best value for a 128gb cuda node even at today's price

holy shit, llama.cpp rpc actually works this is 4x dgx sparks hosting ling 3.1 flash, a 560b moe, at q4 across all four gb10s, about $28k of sparks at today's $6,950 a box no one shipped an official fp4 yet, so i had to try the gguf over rpc and it's possible. it's super slow because of the engine, but i really like the model's output, it has weight to its answers i couldn't stop myself from running this one, i ran ling 3.0 flash on a single dgx spark before qwen 3.8 took over, and this one feels like a real improvement, but it doesn't fit on my 2x dgx sparks, so @niklaslenz_ai, a friend here on x, lent me his 4x dgx spark just to run it it took about 12 hours to complete a build and i'll share it next, stay tuned anon, looks like @AntLingAGI just had an ai leap