kzn.eth - (@overflow_kaizen) header
kzn.eth - (@overflow_kaizen) profile picture

If I have helped you, go out and help others.

/usr/sbin/kaizen
Joined May 2021
Photo by kzn.eth - (@overflow_kaizen)Photo by kzn.eth - (@overflow_kaizen)Photo by kzn.eth - (@overflow_kaizen)Photo by kzn.eth - (@overflow_kaizen)Photo by kzn.eth - (@overflow_kaizen)Photo by kzn.eth - (@overflow_kaizen)Photo by kzn.eth - (@overflow_kaizen)Photo by kzn.eth - (@overflow_kaizen)Photo by kzn.eth - (@overflow_kaizen)Photo by kzn.eth - (@overflow_kaizen)
Alhamdulillah for the 💵 Duit raya 😝
submitted another bug last week at google. Lets wait and see.
6
5
1
101
53,306
hi guys help me, tele i kena hack and ada orang boleh access gambar freehair i dalam tu, since i simpan gambar dalam tu, dia ugut nak leak kan gambar tu, patut ke i reply dia tanya dia nak apa, dia chat kat tiktok then i dah blocked pastu dia email i lagi. dia tahu all my details
8
7
1
93
12,265
kzn.eth - retweeted
Scumbags. This is a large reason why we pay our international employees in Bitcoin.
I'm an Ethiopian founder who won $10,000 AWS Global Innovation Award. Today, @Wise closed my personal account with $9,000 of my money in it. am now unable to pay next month's rent because I can't access my own money. @Wise, I'm asking you to return my $9,000.
27
124
1
1,453
174,508
kzn.eth - retweeted
6
94
7
969
17,800
kzn.eth - retweeted
An NVIDIA RTX 3090 with just 24GB of VRAM is reportedly serving 64 concurrent Qwen3.8-27B sessions at a combined output throughput of 1,035 tokens per second. Even more interesting, the GPU is running with a power limit of just 250W. The experiment uses HyperQwen, an open-source inference-serving stack built around vLLM, to turn a consumer GPU into a multi-user AI server. Most local inference benchmarks focus on how quickly a model responds to one person. This experiment explores a different problem: How much combined generation capacity can one consumer GPU deliver when dozens of users share the same model? That’s an important distinction because serving one conversation and serving 64 simultaneously require very different optimisation strategies. Let’s look at how the setup works. 1. The hardware and reported performance The reported configuration is: GPU: NVIDIA RTX 3090 VRAM: 24GB Power limit: 250W Model: Qwen3.8-27B Inference stack: HyperQwen + vLLM Concurrent sessions: 64 Single-session decode speed: Approximately 127 tokens/s Aggregate output throughput: Approximately 1,035 tokens/s Configured maximum context: Up to 150K tokens Two figures stand out: 127 tokens/s and 1,035 tokens/s. But they measure different things. The approximately 127 tokens/s figure represents reported generation speed in a single-session test. The 1,035 tokens/s figure is the combined output throughput across multiple sessions. It does not mean every user receives 1,035 tokens per second. It also doesn’t mean all 64 users independently generate at 127 tokens/s. The GPU’s compute resources and memory bandwidth are shared across active requests. Individual performance depends on prompt length, context size, scheduling and the number of competing sessions. The achievement is the reported aggregate throughput, not individual user speed. 2. How does one RTX 3090 handle 64 sessions? Serving one AI conversation is relatively straightforward. Serving dozens simultaneously introduces several challenges: Scheduling incoming requests. Managing memory for active conversations. Allocating KV cache efficiently. Keeping the GPU occupied. Preventing memory pressure from limiting concurrency. This is where vLLM becomes important. Two technologies are particularly relevant: Continuous Batching and PagedAttention. Continuous Batching: Keeping the GPU busy Imagine 64 people sending requests to the same AI server. They won’t all receive responses of equal length. One person might ask a simple question requiring 30 tokens. Another might request a detailed coding solution requiring hundreds or thousands. With traditional static batching, requests are grouped together and processed as a fixed batch. When some requests finish earlier than others, resources can remain underutilised while the remaining requests continue. Continuous batching takes a more dynamic approach. Instead of waiting for every request in a batch to finish, the inference engine updates the active batch as individual requests complete. When one conversation finishes generating, another waiting request can take its place. This helps keep the GPU processing active requests instead of wasting capacity waiting for longer responses. For a server handling dozens of users, this can substantially improve aggregate throughput. PagedAttention: Managing KV cache efficiently The second challenge is memory. Every active transformer session requires a Key-Value cache, commonly called the KV cache. It stores attention information from previously processed tokens, allowing the model to reuse information from earlier parts of a conversation during generation. As context grows, KV-cache memory consumption increases. Now consider managing dozens of conversations on a GPU with only 24GB of VRAM. Memory allocation quickly becomes a major concern.
6
13
3
71
9,205
Israel is automatically creating a transcript of every conversation near an Apple device.
239
1,542
130
7,257
378,411
kzn.eth - retweeted
Replying to @yunta_tsai
A petition is underway to reinstate Jensen and I’m signing it too
7,502
20,242
3,133
228,914
10,915,989
GPUs were promised to them 3000 years ago
BREAKING: Nvidia CFO Colette Kress takes over as interim CEO after Jensen Huang resigns, effective immediately.
23
482
1
3,637
26,226
kzn.eth - retweeted
The interim CEO of Nvidia is an Israeli asset. The top 3 candidates to permanently takeover the company are all Israeli. I know Israel does what it wants, but this is so f—king brazen.
503
5,529
387
22,999
704,236
This is why they went after him 👇
For my first post, I’m sharing a letter @NVIDIA signed on why open models matter. AI will transform every industry, power every company, and be built by every country. Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty. The world needs both frontier closed models and frontier open models. images.nvidia.com/pdf/Open-W…
41
466
15
3,729
154,234
黄老板,你是斗不过犹太共济会的。 不过你可以在彻底被橄榄之前,直接和他们爆了。 明天就开源CUDA,把桌子掀了。 这样即使你陨落了,全世界的开发者们也会记起你的好。
107
825
25
10,578
327,645
kzn.eth - retweeted
Sam Altman is the only tech CEO who is 100% safe from Chinese female spies as he isn’t into women I’m bullish on OpenAI
479
617
252
14,951
1,160,179
Manakan tumpahnya kuah kalau tidak ke nasi
Bagus nya system di Malaysia ni... dah curi duit berbillion boleh duduk rumah
11
225
1
749
17,299
Never underestimate twitter or reddit autists when it comes to counterintelligence
Is Huang Xiaole (黄小乐) aka Alice Huang a 燕子 working for the CCP?
73
4,467
60
51,170
1,041,633
kzn.eth - retweeted
Replying to @iamjunaa
Monarchy never intends to save the country. They intend to save their own elites...
16
202
8,972
You know things are getting serious when MyCERT rises from the dead to start issuing updates again. Their last post was in 2019. Let that sink in.
46
2
127
4,530
kzn.eth - retweeted
Early bird tickets are now available lnkd.in/p/e6V5--Nm
5
1
4
541
kzn.eth - retweeted
终于破案了……之前我就一直纳闷,市面上哪来那么多便宜的 Pixel 和三星定制号。 刚才看到 有人在GitHub 直接把做号源码给公开了,谷歌这波羊毛简直离谱。 根本不需要真机,就是把: 12个月基础 + 三星送的6个月 + Pixel送的6个月 三层羊毛硬塞进同一个号,直接叠满 2 年的 Google AI Pro…… 号商的锅直接被端了。 github.com/worldonlyu/Google…
9
34
1
382
42,382
kzn.eth - retweeted
Man in Malaysia was fleeing police — but STOPPED to pay the toll fee The cops didn't pay
48
412
43
3,674
233,291
kzn.eth - retweeted
> Railway gets caught hacking competitor's infra > “sorry, my agents accidentally did it” This is insane
Hey @Railway, is there a reason employee's of yours are trying to breach my infrastructure? Your employee Jake Runzer, just attempted to tonight.
192
1,153
108
24,421
1,108,830