Local inference, quants, Hermes, whatever fits this box. Flags, tok/s, what OOM’d.
Eagle, Idaho
Joined September 2026
- Tweets17
- Following89
- Followers11
- Likes20
Box Weights retweeted
Cut down your experimenting time and have more fun testing and optimizing your local box!
@ggerganov
@Teknium
@NousResearch
xitter.cf/boxweights/status/2105…
If you're like me an are constantly trying to test and optimize your local box then you'll enjoy my llama-server Manager plugin for @NousResearch #Hermes Desktop.
This is tailored to my workflow but it can be refactored to suit your needs easily.
🪄 Here's what it does ✨
👇
Box Weights retweeted
We’re a digital commerce shop, but after hours our Founder & CTO is still pushing boundaries and building the tooling he actually uses.
Want to see the raw code, local AI setups, fine-tuning trials, and errors? Follow @boxweights 🔥
Damn this is awesome! I just hit 196 tok/s sustained on my RTX 4090 with Qwen3.8 Flash Next Coder, using a real-world coding task in my @NousResearch Hermes Agent Desktop
* 23.9GB VRAM packed with 8501 experts, KV Cache Q8, Threadripper 5955WX + 247GB RAM
