Ctrl Stack (@ctrl_stack) header
Ctrl Stack (@ctrl_stack) profile picture

Not chasing every new model. Running local AI for real work on a 7-year-old desktop.

Joined January 2015
Unplugged my old PHP server the morning PHP2Shell dropped. This week my local model and I finally patched it. Plot twist: it was already patched. My distro had backported the fix. php -v doesn't tell you that. The package changelog does. So we called it done. Then I asked "are we actually done?" and it got worse: - The TLS cert had been expired for five weeks. Renewal ran the whole time and failed every time, because the dynamic DNS record was dead. Nobody noticed, including me. - Even with DNS fixed, it would have kept failing. My own firewall only allowed 80/443 from the LAN, so every Let's Encrypt packet would have been dropped. - The weekly backup never ran. Not once. A stray character I typed into the cron file made cron ignore the whole thing. Every backup I had was one I'd run by hand. - 11 old .bak files were sitting in the webroot, serving source code to anyone who asked. None of that showed up on the first pass. It showed up from a restore drill instead of a file listing, a live probe page instead of php -v, and one more round of "what did we miss?" All of it on a 7-year-old desktop with a 16GB card. Local model, my existing SSH keys, nothing sent to the cloud. That box has no business doing this, but it does it anyway.
1
69
Replying to @coffeecup2020
Yep, 2x32 is my current build. I think my MB supports 2x64, but I think those kits are pretty pricy. Need to check though I guess. 馃
6
Replying to @coffeecup2020
I'm going to follow your build closely! Excited for you to get this up and running! Sadly my limitation is only 2 RAM slots on my MB. Otherwise I'd buy the same kit as you right now. Best of luck with the rest of the build!
1
28
Qwen 3.8-flash-next IQ3_XXS at ~50-60 tok/s.
1
12
Replying to @net_termina
How would you recommend building a similar style architecture but for someone who is limited to one model at a time? Different profiles in a harness? Or unload/load models as needed for specific tasks? Asking for the budget/consumer GPU folks out there like myself.
1
26
Replying to @coffeecup2020
Don't believe you need DDR5 RAM. 64GB+ yes, but DDR4 works just fine. I have a 7 year old rig and the only new part is a 5060Ti 16gb. 64GB of DDR4. An AMD 3700x. Mid tier desktop from "forever" ago. And I'm doing useful personal work on it. So budget is doable on DDR4!
4
22
955
I'll read the patch notes to be safe. Thanks for the heads up!
21
Replying to @coldniko
Well now I gotta run to my computer and grab the update. Keep up the fantastic work! So thrilled my 5060Ti can do real work thanks to Strata! 馃
2
6
768
Replying to @0xSero
Oh I thought you were building your own data center. Apologies if I misunderstood.
1
563
Replying to @0xSero
This is fantastic! But also low-key so jealous, weeping in budget hardware land. Mad respect and excited to see how it turns out!
1
2
2,248
Replying to @ashxhart
Do you have numbers for this on a 5060Ti? I'd love to help contribute and bench it for you. Custom kernel, so maybe it won't port to a 5060? Regardless, super impressive and congrats! 馃
210
My hat is off to you for having the capital for 2x RTX 6000s and that much RAM. That's some serious cash and power in a single box. Someday I hope to have even 1x RTX 6000, but I have my doubts. 馃槀
16
Replying to @norsyx
I'm running on a 7 year old desktop with the 5060Ti. Strata is how I'm getting by. Qwen 3.8-flash-next IQ3_XXS at~50-60 tok/s all the way up to 200k context. But I have 64gb of system RAM that helps out with the offloading. But I bet you could also get some great speeds at 32gb!
1
2
175
The most reliable method to run locally on my 5060Ti is using Strata by @coldniko. Running Qwen 3.8-flash-next IQ3_XXS at ~50-60 tok/s. Needs about 45GB of my 64GB system RAM. But it's been a powerhouse and I couldn't be happier!
2
74
Replying to @areimann
I'm running on a 7 year old desktop, only new hardware is a 5060Ti 16gb. My main use case is a personal wiki. End of each day I record a voice memo of what I accomplished, blockers, in progress, up next, etc. Helps me decompress and frees up my mind to think about the rest of life without trying to keep track of all my work projects and notes. Using Strata with Qwem 3.8-flash-next. Absolutely loving it!
1
63
Replying to @ItsmeAjayKV
Strata and Qwen 3.8-flash-next has already unlocked "frontier" level intelligence on my 7-year old desktop. Granted, "frontier" from ~10 months ago. But Opus 4.6 was groundbreaking at one point. Now it's obsolete... amazing how tech rapidly advances.
1
2
93
Replying to @ItsmeAjayKV
If this is true, I will be beyond thrilled. A 27B punching at better than Sonnet 5.5 (high)... why ever pay for an API bill again?
1
6
793
Replying to @Hikari_07_jp
The waiting is killing me. And probably everyone else too. 馃槀
1
244
Replying to @hotdropfps
Ryzen 7 3700X & 5060Ti 16GB
65
Replying to @coldniko
Let's go! Qwen 4 here we come!!!
1
383
Replying to @coldniko @UnslothAI
So excited to try these later today! Been running Swift IQ3_XXS as my go-to, but these I have to try out! 馃馃
1
374