Vik | Practical AI & Tech (@uctive) header
Vik | Practical AI & Tech (@uctive) profile picture

Private local AI server for small businesses: your data stays inside your company. https://teapawt.org/t.co/dnhFJ2Yoqh

Como, Italy
Joined December 2011
Photo by Vik | Practical AI & Tech (@uctive)Photo by Vik | Practical AI & Tech (@uctive)Photo by Vik | Practical AI & Tech (@uctive)Photo by Vik | Practical AI & Tech (@uctive)Photo by Vik | Practical AI & Tech (@uctive)Photo by Vik | Practical AI & Tech (@uctive)Photo by Vik | Practical AI & Tech (@uctive)Photo by Vik | Practical AI & Tech (@uctive)Photo by Vik | Practical AI & Tech (@uctive)
Vik | Practical AI & Tech retweeted
Bit by bit
🚀 Strata v0.2.14 is out! Released today, November 15, by @coldniko (Niko Veit). This is a major architectural milestone: Strata now runs prompt-expert computation on its own native GPU kernels, replacing the previous llama.cpp path on NVIDIA and AMD GPUs (Windows/Linux). And the benchmarks are impressive: 🔥 RTX 5090: 7,870 vs 6,851 tok/s — +14.9% prompt processing 🔥 RTX 4090: 20K-token prompt in 5.65s vs 6.11s — ~7.5% faster 🔥 RTX 5070 Ti: 1,408 vs 953 tok/s — +47.7% on a 20K-token prompt 🔥 Dual RTX A6000: 75.88 vs 64.16 tok/s — +18.3% output with improved multi-GPU memory allocation 🔥 Mac M5 Max: 94.7 vs 92.8 tok/s output with IQ3_S. No new settings. Existing models, chats and configurations remain intact. What makes this release particularly interesting isn’t just the speed improvement. Strata is progressively replacing inherited inference components with its own implementations. An independent inference engine is taking shape, release by release. Huge respect to Niko for the pace of development. We’re looking forward to testing this version on our own hardware! 👨‍💻 Niko: 🔗 GitHub release: github.com/Niko1221/Strata/r… teapawt.org/coldniko
32
11
3
261
18,429
Vik | Practical AI & Tech retweeted
My recommended $1,000 V100 + Strata rig build! (Note: 2× 16GB gives 1.7× the prefill speed compared to a single 32GB). 100% second-hand hardware. Swap to DDR3 RAM and the value is absolutely unbeatable. 🔥
29
20
5
325
32,320
Vik | Practical AI & Tech retweeted
I’ve worked that heatshield my entire adult life. Peak career moment today To go from little 7 tile patches to a full reusable ship. Absolute marathon I’m so grateful I got to do something useful, that I was given the chance I don’t really have many good memories from this experience. It was so difficult. It almost looks easy now, but none of it came easily Please do something amazing with this rocket. This tech is such a gift, a love letter to the species
1,115
2,567
171
31,432
555,878
Vik | Practical AI & Tech retweeted
Here's a budget localmaxxing find. I've been keeping this one to myself because I don't want it to sell out. Could be a fun holiday gift for someone who wants to start experimenting with local AI. This is the Origimagic C7 mini PC - ~30% off. I paid $289 for 16GB DDR4, a 512GB SSD, and two Thunderbolt 4 ports. I added another 16GB for $89, bringing mine to 32GB RAM for $378 total, before the eGPU. It’s the only cheap Thunderbolt 4 PC I’ve found that checks my boxes, and it works with at least one Gigabyte eGPU. Next experiment - seeing whether I can run two eGPUs on this thing. Currently $299 for the base machine. Add another 16GB for $100 Anyone else running a mini PC + eGPU localmaxxing setup? Affiliate links in the thread.
26
13
4
323
49,438
Vik | Practical AI & Tech retweeted
😯Why not buy a $702 AMD GPU from 2018 (~1TB/s memory bandwidth) with more VRAM than a $1,790 RTX 3090? And it's running Qwen3.8-27B at 8-bit precision. A local AI enthusiast has been tracking used AI GPU prices on eBay through a new tool called RackBites. Here are the most recent numbers. GPU → VRAM → Low-end asking price 🟢 Tesla P100 → 16GB → $174 🟢 Tesla P40 → 24GB → $280 🟢 Tesla V100 → 16GB → $220 🔴 AMD MI50 → 32GB → $702 🟢 Tesla V100 → 32GB → $949 🟢 Quadro RTX 8000 → 48GB → $2,111 🟢 RTX 3090 → 24GB → $1,790 🔴 Radeon AI PRO R9700 → 32GB → $2,430 But the AMD MI50 really caught my attention. 🔴 MI50 32GB 💾 HBM2 memory ⚡ ~1TB/s memory bandwidth 💵 $21.93 per GB of VRAM And people are still optimizing local AI software for it. Check out a recent single-GPU benchmark using a specialized llama.cpp fork 🧠 Qwen3.8-27B 💾 Q8_0 weights + Q8_0 KV cache ⚡ 2K context → 22.8 tps ⚡ 64K context → 20.4 tps 🔥 64K reasoning + MTP → 28.8 tps 💻 Short coding + MTP → 51.8 tps On a GPU released in 2018! Now here's another interesting detail. The RackBites developer bought an 8× V100 32GB server in early September for $5,400. 256GB of GPU memory. The same server is now reportedly listed for $13,000. Are older datacenter GPUs becoming more valuable as Localmaxxers find new ways to run modern models on them? These are eBay asking prices, not completed sales. Links in replies
31
17
5
199
24,647
Vik | Practical AI & Tech retweeted
Starship flight 15 has just successfully lifted off 🚀
46
139
15
1,919
105,712
Vik | Practical AI & Tech retweeted
You know how I work with vulns at my job? Let me tell you. I run dozens of subagents on DeepSeek V4.1 Flash Abliterated. Each one pushes 350-400 tok/s. All of it is self-hosted on our own bare metal. And it is only dozens because I do not have more vuln tasks than that. Give me more work and I just start more agents. The hardware is not the limit. Why abliterated? Because to a safety filter, security work looks exactly like an attack. When Hugging Face got hit by an AI agent attack this year, its security team tried the top commercial models first. The models refused to look at the attack. Why self-hosted? Our code and our findings never leave our servers, and nobody can rate limit us in the middle of a scan. That is 1 guy with a small queue of tasks. Now imagine what the big players are running. State teams. Big security companies. People who never post about it. They are not waiting for the debate about which models Hugging Face should delete.
17
18
162
5,603
Vik | Practical AI & Tech retweeted
Big new open-weights model! This one could be a top model. Check out Code Arena scores. Beats DeepSeek V4.1 and Kimi K3 in coding Baidu released a 280B MoE model that activates only 14B parameters per token. And it's already challenging DeepSeek and Kimi in coding. 🇨🇳 This is ERNIE 5.5 Flash Preview. Model Stats 🧠 280B total / 14B active 📚 1 MILLION token context ⚡ Baidu reports 200+ tokens/sec on its hosted platform 🔧 Coding, tool calling and agentic workflows 🔓 Apache 2.0 weights coming soon Now look at the early Code Arena WebDev scores: 🔥 ERNIE 5.5 Flash Preview → 1668 🟢 DeepSeek V4.1 Flash (Max) → 1664 🔵 Kimi K3 (Max) → 1658 These are early results, with ERNIE's score based on Arena's automated evaluation rather than human votes. Baidu is showing ERNIE working across entire agent workflows like these examples ... 🎮 Built a third-person Unreal Engine 5.8 game in an 11-hour coding session. 🕹️ Built nearly 100 mobile mini-games from design documents. ✈️ Two agents collaborated to build an airport and Boeing 747 in Blender. ⚙️ It created an editable W16 engine in FreeCAD. 📊 Produced research reports, spreadsheets and presentations. How much hardware will we need? Raw weight math for 280B parameters: 💾 BF16 → 560GB 💾 8-bit → 280GB 💾 4-bit → 140GB 💾 3-bit → 105GB 💾 2-bit → 70GB Before quantization metadata, KV cache and runtime overhead. ⚠️ API available now. The post-trained Preview weights are NOT downloadable yet. Baidu says Apache 2.0 weights are coming. Link in ALT
22
7
1
133
11,874
Vik | Practical AI & Tech retweeted
I tested 8 AI models on 12 questions from my own field. Every question 3 times. Textbook questions: 96 % correct. My real paperwork: 4 %. Times a model said "I don't know": 0 of 288. Full results: os-ai.thomasbaertl.com/not-o…
1
2
26
Vik | Practical AI & Tech retweeted
I built an X-ray mirror using Claude Fable 5.2. 💀 Point a webcam at your face and see the skull, brain and muscles inside your head, tracked live as you move. Open your mouth and the jaw opens. Hold up a hand and you see its bones. One web page, no install, and the video never leaves the computer. Built with Claude Fable 5.2 by describing what I wanted. You can download it and try it for free. Link in the comment section. 👇🔗 Anatomy models: BodyParts3D, © DBCLS, CC BY-SA 2.1 Japan.
85
203
44
1,376
82,352
Vik | Practical AI & Tech retweeted
я знаю, куда я потрачу 139$ в новом году
5
1
1
56
10,987
Vik | Practical AI & Tech retweeted
Apple is working on a new app called Harbor Harbor turns your Mac into a private AI inference server. Run open-weight models locally on Apple silicon—your data never leaves your hardware, and nothing reaches the cloud. Connect multiple Mac computers over Thunderbolt 5 to pool unified memory, enabling large models and higher concurrency that a single computer can handle alone. Set up in minutes, not hours Harbor automatically detects your devices over Thunderbolt 5 to form a cluster. Plug in, start, and you’re running—no manual configuration, no command line, no infrastructure expertise required. Download and run models directly Browse, download, and load open-weight models from within the app. Harbor handles model management so you can focus on the work, not the setup. Cluster for scale Connect two or more Thunderbolt 5-capable Mac computers and Harbor pools their unified memory into a single inference resource. Run models that exceed any single computer’s capacity, or serve more concurrent requests across your cluster. Harbor is currently in a private preview under the Claris brand.
243
504
242
5,995
766,480
Vik | Practical AI & Tech retweeted
Starting to roll out today: Claude Whiteboards, Claude Sheets and Claude Data. Think a plan through on a board, build a tracker in a sheet, and tell a story with your data.
1,548
5,274
3,860
64,352
12,400,129
Vik | Practical AI & Tech retweeted
BOOM 🚨 Elon Musk got a simple question today: when is Grok 4.8 coming? 🔥 And his answer turned into one of his nerdiest threads in a while 🔥 First came the reason for the delay. SpaceXAI ditched JAX and SGLang, two popular open-source tools for training and running AI models, and rewrote Grok’s entire RL and inference stack from scratch in C/C++. RL, or reinforcement learning, is the stage where the model practices and gets graded on its answers. Inference is the part that actually runs Grok when you ask it something. Then the replies kept coming, about C, about memory and even about assembly. The most cryptic one was this: “At SpaceX, we write our own headers and calculate our own time (includes leap seconds).” Elon warned that not many people would know what it means. He also said the new Grok stack is derived from the work behind Starlink. So here’s what he meant in plain English, and why it explains the extra time Grok 4.8 needed 🐰 First, some quick context: the rewrite didn’t start today. In September, Elon said the Starlink software team had already rebuilt the pre-training stack in C and C++, by hand and not by AI. That’s the software behind the huge first phase, where the model learns from mountains of data. Back then, the RL side was still a legacy pile of third-party software. Now that pile is gone too. Grok 4.8 is in RL, and a soft rollout to Grok Bot users is coming later this week. Elon promises 4.8 will bear no resemblance to 4.7, because this time the foundations are built right. Those foundations start with the smallest building blocks. 📚 DECODING “WE WRITE OUR OWN HEADERS” In C, which Elon says makes up most of the new stack with “a little C++, but not too much,” a header file lists the building blocks a program plans to use. Think of it as the parts catalog the code checks before it gets to work. Most teams grab the standard catalogs that ship with their tools and trust them. Writing your own most likely means SpaceX defines those parts itself, so nothing in the code does anything the team didn’t choose. Memory gets the same treatment. When someone asked whether the old stack was written in Rust, Elon fired back: “Real men manage their own memory.” That jab has history behind it. xAI’s earlier Grok-1.5 stack was built on JAX, Rust and Kubernetes. Python tools like JAX, and even Rust, take care of much of the memory bookkeeping for you. In C, programmers decide exactly when every chunk of memory gets claimed and released. It’s more work and far less forgiving, but nothing happens that the team didn’t ask for. Time is where that control gets really interesting. ⏱️ THE EXTRA SECOND MOST COMPUTERS FAKE AND SPACEX COUNTS Atomic clocks keep near-perfect time. The Earth’s spin doesn’t. To keep the two in step, timekeepers have added 27 leap seconds since 1972, most recently on December 31, 2016. On those nights, the official clock actually reads 23:59:60 before rolling over to midnight. Linux and Unix assume every day is exactly the same length, so they have to fake that extra second. A common trick is to repeat a second, which means the clock briefly steps backward. That single second has a rap sheet. In 2012, a Linux bug triggered by the leap second took Reddit down for about 40 minutes and hit LinkedIn. Qantas staff had to check passengers in by hand, and one sysadmin reported spending the night rebooting hundreds of servers. In 2017, it was Cloudflare’s turn. Its DNS service, which works like the internet’s phone book, timed how long each lookup took. When the clock stepped back, the result came out negative and the software panicked. Cloudflare later admitted the root cause was simply believing that time can’t go backwards. Big Tech’s answer was a clever workaround. It’s called the smear. Google and Meta spread the extra second across many hours (Meta uses 17) by running every clock just a hair slow, so 23:59:60 never shows up. It keeps servers calm. The catch is that for those hours, every clock is deliberately a little off from official time. That’s fine for a website. It’s a much tougher sell when your hardware moves about 7.6 km (4.7 miles) every second, which is roughly how fast a Starlink satellite flies. At that pace, even a fraction of a second covers serious distance. SpaceX also juggles two clocks at once. GPS never added leap seconds, so it now runs 18 seconds ahead of UTC, the time the rest of the world runs on. Any software that mixes the two needs the exact gap for every date in its records, because each of those 18 jumps landed on a specific night. So SpaceX doesn’t hand time off to a library or smear it away. According to Elon, it counts every single second itself. 🧠 HOW STARLINK’S HABITS ENDED UP INSIDE GROK Starlink is basically a giant computer network that happens to live in orbit. Back in 2020, with only about 480 satellites up, SpaceX’s Starlink software lead said the constellation already carried more than 30,000 Linux computers. He said the team treats satellites like servers in a data center, and that running hundreds of them 24/7 will find edge cases in every system. A frontier AI training run is the same kind of beast on the ground. Tens of thousands of GPUs have to act as one machine for months, and a single rare bug can stall the whole run. Leap seconds won’t make Grok any smarter, but the habit behind them might. There’s even some assembly in the new stack, the raw instruction language a processor actually speaks. At that level, knowing exactly what every layer does can turn into real speed. That isn’t just Elon talking. In September, Google software engineer Patrick Toulme pointed to how efficiently the C++ stack uses its GPUs as part of the reason xAI keeps churning out models so fast. Elon also owned the irony. One day superintelligence will write a better software stack, he said, but that day hasn’t come yet. For now, Elon is trusting human engineers with the code that matters most. The timing makes the whole story even better. On October 15, the world’s timekeepers voted to stop adding leap seconds starting May 20, 2027. A big reason was the growing chance that Earth’s faster spin would one day force the first-ever negative leap second, which critical infrastructure is largely unprepared for. Meta had been pushing to kill the leap second since at least 2022. The vote doesn’t erase the 27 leap seconds already added since 1972, though. They stay in the record for good, so any software that reads old dates or converts between GPS time and UTC still has to count them. Big Tech spent years pushing to make that troublesome second disappear. SpaceX simply did the math, and those same engineers are now writing the software behind Grok 4.8! Elon admits 4.8 isn’t incredible yet. What excites him is the foundation, which he expects to evolve into something special. If he’s right, every Grok after 4.8 will grow from code where the team knows exactly what every byte of memory and every second of time is doing. And the early signs are pointing up. Elon says 4.8’s evals are already starting to climb in RL, and this time the climb begins on a foundation SpaceX built down to the last second.
Replying to @elonmusk @tetsuoai
At SpaceX, we write our own headers and calculate our own time (includes leap seconds). Not many people will know what this means, but those that do will appreciate it. The new Grok AI/SI training and inference stack is derived from incredible of work done by amazing engineers to create Starlink.
24
55
9
720
91,268
Vik | Practical AI & Tech retweeted
Replying to @yasakani_dev
楽しみにしています。 Strataとの比較されてますでしょうか。
1
1
5
12,512
Vik | Practical AI & Tech retweeted
Me running to Fable 5.2 with another startup idea that will make $0.
99
235
30
4,212
178,435
Vik | Practical AI & Tech retweeted
🔴5 farklı yapay zeka ajanı 145 m2’lik bir dairenin tüm mekanik ve elektrik tesisatını tasarladı Her agent farklı bir sistem üzerinde çalıştı: temiz su, pis su, havalandırma, elektrik ve yangın tesisatı Toplam 4871 farklı tasarım denendi. Sistemler birleştirildiğinde 11 tesisat çakışması tespit edildi. 6. yapay zeka ajanı devreye girerek tamamını giderdi. İlk tasarımda 139 saniye olan sıcak su bekleme süresi ise 6 saniyeye düşürüldü. Mause imleçinin ne kadar hızlı çalıştığını görebiliyor musunuz?
68
285
91
3,054
938,854
Vik | Practical AI & Tech retweeted
I already see clear signs that who is refusing AI assistance for programming it is starting to lag behind in a significant way, where this means: they are living in a bubble where losing work will happen faster compared to the other folks.
63
42
13
869
44,533
Vik | Practical AI & Tech retweeted
I suspect we're entering the temporary golden age of what a regular dude can do with AI. Enjoy it while it lasts.
96
174
38
3,589
80,665
Vik | Practical AI & Tech retweeted
You don't need a GPU to run a frontier-size AI model anymore. llama-modchip runs DeepSeek's 284B model entirely in your computer's RAM: - The full 284B model fits in 64 GB of RAM - 0 GPUs. Just your CPU - About 9 to 15 words per second - A 16 to 32 GB laptop runs 30B models at 25 to 60 tokens per second - Works on Intel, AMD and Apple chips A model this size used to need a rack of GPUs. github.com/synchopate/llama-…
25
34
1
216
63,822