WPPA (@WPPA) header
WPPA (@WPPA) profile picture

WPPA

@WPPA

CLONE GOD AND QUALIFIED EXPERT ON EVERY TOPIC. B01E95BEA77F2C21AC223CEFAB4378F619AD8F15 @ https://teapawt.org/t.co/Ck5TeMrbjG

Joined February 2010
  • Tweets718
  • Following31
  • Followers114
i am absolutely speechless at @coldniko's Strata project. from an average of 5 TPS on llama.cpp to an average of 50 TPS on Strata with Qwen3.8 Flash Next. this is faster than Qwen3.8 27B on the same hardware. amazing.
1
1
90
Qwen3.8 Flash Next GSQ RCO Coder IQ1_M on a single 3090 and 32 GB of RAM at 14.7 tokens per second decode
59
why would they design computers that look like the twin towers
Image from WPPA (@WPPA)
14
what is the config setting to prevent exllamav3 from increasing the vram usage over time for seemingly no reason? i can fit 256,000 context into 24 GB VRAM but after a few messages it increases usage past that and becomes unusable.
1
12
ive decreased the context window down to 175,104 and it still does the same thing. 21 GB used after load, 26+ GB and leaking into system memory after a few prompts.
9
how do GPU power limits even work like i limited my 3090 to 75% power and it is still getting 50 tokens per second on Qwen3.8 27b
18
do yall remember when this guy had a stranglehold on ogu and put the entire com in terror
Lorenzo Franceschi-Bicchierai (@lorenzofb) profile pictureLorenzo Franceschi-Bicchierai@lorenzofbSep 21
~ Personal news ~ I left TechCrunch. I will now focus on finishing my book about Hacking Team and the history of government spyware. After that, and in the meantime, I will be freelancing. Contact me: [email protected] or Signal @ LorenzoFB.1337 (+1 917 257 1382)
Image from Lorenzo Franceschi-Bicchierai (@lorenzofb)
13
i did not win the silicon lottery my GPU can't even be undervolted it just crashed with even "safe" values
5
nevermind, it seems to have an issue with memory usage growth. starts out using 22 GB VRAM and then grows to 26 GB VRAM over time and beyond which my card can't hold so it's falling back to system memory.
WPPA (@WPPA) profile pictureWPPA@WPPASep 21
how long does it take on a fresh reinstall for exllamav3 to stop triton compiling and actually output at usable speeds? right now i'm at about 9 TPS on a 3090.
12
how long does it take on a fresh reinstall for exllamav3 to stop triton compiling and actually output at usable speeds? right now i'm at about 9 TPS on a 3090.
1
19
what is the secret sauce for qwen3.8 27b exl3 for a single 3090? i have immense difficulty with it losing intelligence or going into thought loops.
1
21
Need to sell my house and buy DGX Sparks ASAP
5
how do i get 200 TPS prefill on qwen3.8 flash next but only 4.30 TPS decode lol
24
you already know i'm snapping up .si domains to catch on the Superior Intelligence wave
Image from WPPA (@WPPA)
2
52
qwen3.8 flash next uncensored by delignai -> outputs gibberish qwen3.8 flash next uncensored by orcarouter -> outputs gibberish qwen3.8 flash next regular by ista-daslab -> works fine what does this mean
51
why does @UnslothAI desktop loading qwen3.8 flash next take up all my VRAM and all my shared memory and all my RAM loading and it's painfully low where as the exact same flags on the packaged llama-server works fine? 🤔
1
48
qwen3.8 flash next on a single 3090 and 32 gb of DDR4 at 128,000 context window prefill: 211 tps decode: 9 tps almost usable?
32
what's the word on if letting a LLM reason infinitely or setting a small reasoning budget (~4096) is better?
11
we need a hugging face torrent archive similar in distribution mechanisms to anna's archive where you have it all in chunked optimized formats with a front end that allows you to select the most desired/most in need of seeding models or collections of models
62
running Qwen3.8 Flash Next at Q2_K_XL in Unsloth Desktop on a single 3090 and 32 GB of pitiful DDR4 2133 MHz RAM. speeds are about what you would expect, 9 tokens per second. but it's working!
1
66
turns out i had suboptimal settings. changed them to the optimal and still it runs at 9 tokens per second.
1
19
this sucks, running it on a code base and we're at 6.5 tps.
9
Load more