@lyesbahi

Président de https://xitter.cf/t.co/cBlYWR3VC5, passionné par l'innovation technologique.

 Paris
sysops.fr
Joined June 2010
  • Tweets451
  • Following88
  • Followers87
121Media
Strata 0.1.51 on my RTX 5090, same-pack A/B vs Project Maya. Below: 28.7/1013 vs 36.2/1333 tok/s - ~26% headroom in the prompt kernels. That speed belongs to everyone running DeepSeek-V4.1-Flash on Strata. Devs, the numbers are yours. I keep testing.
Benchmark table ( RTX 5090 32 GB · 121 GB RAM · DeepSeek-V4.1-Flash-1017, IST-DAS 2.25bpw IQ2_XS pack · single user, warm cache, Oct 25 2026)

ALT Benchmark table ( RTX 5090 32 GB · 121 GB RAM · DeepSeek-V4.1-Flash-1017, IST-DAS 2.25bpw IQ2_XS pack · single user, warm cache, Oct 25 2026)

6
3
1
35
3,371
"Strata is 1.8x slower on DeepSeek" - I measured it, and I was right. Until one flag: --mmap-experts off → 12.9 → 28.6 tok/s, now faster than Maya on decode. Same code, same box, same weights. The bug wasn't the engine, it was the engine's guess about your RAM. below the table
Strata 0.2.4.2, IST-DAS 2.25 bpw · auto = streams from NVMe · --mmap-experts off = copies 11,666/12,983 experts into RAM (113 GB) · RTX 5090, 121 GB RAM, 24 Oct 2026.

ALT Strata 0.2.4.2, IST-DAS 2.25 bpw · auto = streams from NVMe · --mmap-experts off = copies 11,666/12,983 experts into RAM (113 GB) · RTX 5090, 121 GB RAM, 24 Oct 2026.

15
16
4
267
18,919
One flag was the whole story: the engine streamed its 274 GB pack from NVMe because "doesn't fit in RAM", while it already had the code to copy in what fits. One flag, +120%.
1
4
2,161
DeepSeek-V4.1-Flash (256 experts, 274 GB) running at double-digit tokens/s on one RTX 5090 and 121 GB of RAM was unthinkable a year ago. Two engines do it today. We benched both - and the lesson is about RAM placement, not about anyone losing
11
4
87
5,523
more replies
And today's real win: same model, one modest PC, two healthy engines racing each other instead of one stack being required. Both projects shipped fixes within 24h when we hit a bug together. This is what an open ecosystem looks like.
1
2
264
Bottom line: Maya wins this snapshot by 1.8x on this box - placement, not engine. Strata's DeepSeek port is one week old and already reads everything, speculates, and runs where others need servers. Watch this space; the gap should shrink fast.
3
3
244
DeepSeek-V4.1-Flash on a single RTX 5090: Maya hits 31.6 tok/s decode; Strata 0.2.4 just ported the engine. Their Qwen MoE runs 185 tok/s on that card, ~40 on 2016 Tesla P40s. Maya vs Strata 🥊 one GPU, two engines - both ship faster than we can re-bench.
Benchmark table comparing two local inference engines on the same hardware, decode speed in tokens per second. DeepSeek-V4.1-Flash (748B MoE) on a single RTX 5090: Maya v1.1.26 at 31.6 tok/s measured, Strata 0.2.4 pending with the 274 GB model still downloading. Qwen MoE on the same RTX 5090: Strata at 185 to 225 tok/s. Qwen MoE on two 2016 Tesla P40 GPUs: Strata at about 40 tok/s. Measured 21–23 October 2026 on our own hardware.

ALT Benchmark table comparing two local inference engines on the same hardware, decode speed in tokens per second. DeepSeek-V4.1-Flash (748B MoE) on a single RTX 5090: Maya v1.1.26 at 31.6 tok/s measured, Strata 0.2.4 pending with the 274 GB model still downloading. Qwen MoE on the same RTX 5090: Strata at 185 to 225 tok/s. Qwen MoE on two 2016 Tesla P40 GPUs: Strata at about 40 tok/s. Measured 21–23 October 2026 on our own hardware.

4
8
137
10,401
Maya, same exercise: DeepSeek (maya-ds4) on my single RTX 5090 + 121 GB RAM: 30-33 tok/s in real use, 25 in the bench, 784 tok/s prefill at 8K. 147.6 GB of experts read in place, CPU lane 11-28%. +15 to +19% since v1.1.9; the DSpark drafter adds nothing.
Table: Maya (DeepSeek maya-ds4) on one RTX 5090 with 121 GB RAM. Decode tokens/s: v1.1.9 on Oct 20 gives 21.1, fp16 19.2, the DSpark drafter 22.1 and 23.4, v1.1.19 on Oct 23 gives 24.2 and 25.1, and live use 30 to 33. Prefill at 2K prompt: 249 to 265; at 8K: 671 to 792. 147.6 GB of experts read in place, CPU lane 11 to 28 percent. Notes: decode up 15 to 19 percent from v1.1.9 to v1.1.19; Strata IQ3_S on the same card reaches 182 to 240 tokens per second.

ALT Table: Maya (DeepSeek maya-ds4) on one RTX 5090 with 121 GB RAM. Decode tokens/s: v1.1.9 on Oct 20 gives 21.1, fp16 19.2, the DSpark drafter 22.1 and 23.4, v1.1.19 on Oct 23 gives 24.2 and 25.1, and live use 30 to 33. Prefill at 2K prompt: 249 to 265; at 8K: 671 to 792. 147.6 GB of experts read in place, CPU lane 11 to 28 percent. Notes: decode up 15 to 19 percent from v1.1.9 to v1.1.19; Strata IQ3_S on the same card reaches 182 to 240 tokens per second.

3
1
36
5,808
9 days, 2 servers (1x RTX 5090 + 2x 2016 Tesla P40), Strata 0.1.44 -> 0.1.48: 5090 solo 81 -> 182 tok/s (quant change), two users 34 -> 93 per person, P40 40 -> 46 (+15% from the NUMA pool). ~1,000 real requests measured engine-side.
Text table comparing Strata decode speeds on two servers, Oct 16-23 2026. 5090 server: solo 80-86 tok/s, then 180 after switching the model to IQ3_S, then 182 on engine 0.1.48; per user at two users: 32-42, 85, 93. Two Tesla P40 server: solo 40-44 on engines 0.1.44-0.1.45.3, 45-46 today on 0.1.48, where the NUMA pool adds about 15 percent.

ALT Text table comparing Strata decode speeds on two servers, Oct 16-23 2026. 5090 server: solo 80-86 tok/s, then 180 after switching the model to IQ3_S, then 182 on engine 0.1.48; per user at two users: 32-42, 85, 93. Two Tesla P40 server: solo 40-44 on engines 0.1.44-0.1.45.3, 45-46 today on 0.1.48, where the NUMA pool adds about 15 percent.

5
349
Strata 0.2.3 (engine 0.1.48) on my RTX 5090: Qwen3.8-Flash-Next IQ3_S hits 240 tok/s solo, 115 per user at two, 256K context. No speed change for this card - 0.2.3's gains are on Strix Halo, Arc and Mac. Its NUMA pool is what gave my 2016 P40s +15%.
3
5
36
3,650
Qwen3.8-Flash-Next (IQ3_S, 84.6 GB) running on 2× Tesla P40 cards from 2016 on Strata 0.2.3 by @coldniko : 46 tok/s solo, ~42 for two users, 256K context. The per-socket NUMA pool buys +15% on a dual Xeon. Old GPUs, new tricks.
5
561
185 tok/s decode (225 peak) running Qwen3.8-Flash-Next on ONE RTX 5090 + 121GB DDR5. IQ3_S GSQ-RCO quant (ISTA-DASLab) on Strata ( @coldniko ). full 262K ctx, experts fully RAM-resident (98.5% cache hit), 9K prompt in ~1.4s. Flags: --kv k8v4 --prefill auto:16384 --spec 4.
2
1
12
1,943
Running Qwen3.8-Flash-Next (125B) on an RTX 5090 using Strata by @coldniko : 484W drawn, but GPU temp stuck at 56°C. Worst room heater ever. Refund please!
1
2
205
49 t/s on 10 year old GPUs. Running Qwen3.8 Flash Next (126B) on dual Tesla P40s: llama.cpp: 2-3 t/s Strata: ~49 t/s Game-changer for legacy hardware. Huge thanks to @QwenDevs for the amazing open models and @coldniko for Strata! github.com/Niko1221/Strata
Strata dashboard showing Qwen3.8 Flash Next running at 49.8 t/s with 100% utilization across dual Tesla P40 GPUs

ALT Strata dashboard showing Qwen3.8 Flash Next running at 49.8 t/s with 100% utilization across dual Tesla P40 GPUs

1
2
102
En un mot! Merci
1
1
37
Load more