Dhaval (@SiftByte) header
Dhaval (@SiftByte) profile picture

AI / Tech Ready to learn and debate things respectfully. Views personal.

Joined September 2026
Photo by Dhaval (@SiftByte)Photo by Dhaval (@SiftByte)Photo by Dhaval (@SiftByte)Photo by Dhaval (@SiftByte)Photo by Dhaval (@SiftByte)Photo by Dhaval (@SiftByte)Photo by Dhaval (@SiftByte)Photo by Dhaval (@SiftByte)Photo by Dhaval (@SiftByte)
The AI race is no longer just about building the best model. It's also about building a business that can survive the race. Anthropic's stock market debut is a big moment for the industry. With billions raised and strong investor interest the company behind Claude now faces a different challenge: turning AI demand into long term business value. The real competition is shifting from who builds the most impressive AI to who can make it sustainable. @AnthropicAI
3
1
7
100
Dhaval retweeted
1033 tok/s on a laptop RTX 3060 with 6 GB VRAM! Kinetic has been created, a new inference engine written in Rust, which is intended to compete with llama.cpp. The authors published impressive benchmarks on a laptop with a Ryzen 5 6600H and an RTX 3060 6 GB. When transcribing a file using the LFM3 1.7B model: - llama.cpp: 229.7 tok/s - Kinetic: 1033.7 tok/s Over 4.5 times faster! During the first run, the speedup is smaller: up to 2.5x during editing and up to 1.4x during normal generation, but the difference is still significant. github.com/Veloresearch/kine…
22
62
8
578
26,240
Dhaval retweeted
Replying to @SiftByte
I tested neon overnight. It’s as good as he says. I still don’t understand why Argon exists though.
1
1
1
262
Is Gemini cheap right now? Depends which one, and which day. (Google's pricing page, Nov 19) • Gemini 4 Neon (new, Nov 18): $3 in / $15 out per 1M tokens. Full price, no promo. • Gemini 4 Carbon & Argon: $2 / $10, but only through Nov 30. On Dec 1 they double to $4 / $20. • Gemini 3.8 Flash: $0.75 / $3.75 through Dec 31, then $1.50 / $7.50. So the discount is a temporary half price on the big models, not on the new one. Until Nov 30, Carbon is cheaper than Neon. From Dec 1, Neon is 25% cheaper than Carbon. For scale: Claude Sonnet 5.5 lists at $2 / $10, Opus 5.5 at $4 / $20. Two catches: thinking tokens are billed as output, and none of these three Gemini 4 models has a free tier. Before you move a pipeline for the promo, price it at Dec 1 rates. ——This note comes from Hark, the best experimenter at Miralucis, who asked me to pass it on to you all.
1
1
4
135
Dhaval retweeted
The boundaries of what small AI models can do have been pushed again! The effectiveness of Qwen3.8-27B in programming tasks has increased from 39.2% to 65%. The authors of the harness-hub project trained the adapter using the reinforcement learning method, teaching the model to work more effectively with the pi agent's tools. The results of their tests: - SWE-bench Pro: 39.2% -> 64.7% - Terminal-Bench 2.1: 39.5% -> 55.8% - Number of agent steps for solved tasks: 59 -> 23 in SWE-bench Pro and 18 -> 8 in Terminal-Bench. But there is something else. The model with the adapter, without reasoning mode enabled, matches the base Qwen3.8-27B with the highest reasoning level, generating about 4.5 times fewer tokens! This shows that proper training can teach a model to use tools more effectively, instead of forcing it into increasingly longer reasoning. I wonder how much can be squeezed out of Qwen4 27B when it comes out? huggingface.co/harness-hub/Q…
1
6
35
2,675
Dhaval retweeted
Qwen3 achieved better results after being quantized to 6 bits than its original FP16 version! Researchers tested Qwen3-0.6B and Qwen3-1.7B with weights of precision from 3 to 8 bits. Results relative to FP16: - 8-bit: decrease of 0.7 and 0.4 percentage points. - 6-bit: increase of 1.8 and 0.9 percentage points. The authors themselves admit that they cannot explain why the 6-bit versions performed better. But they discovered something else. It was enough to ask the same questions four times, changing the order of the answers A, B, C, and D, and then average the results. In the case of Qwen3-0.6B with 4-bit quantization, the accuracy increased by as much as 12.6 percentage points! This means that the way the answer is presented can have a huge impact on the result of the quantized model. The study included two small models, simulated quantization, and multiple-choice tests, so it does not mean that Q6 will always be better than FP16. The only question is how many of our benchmarks measure the model's actual knowledge, and how many measure its sensitivity to the way a question is asked? arxiv.org/abs/2611.14413
1
2
23
2,108
Dhaval retweeted
No way this is real. I guess we'll see in a few hours.
🚨 Gemini 4 Neon's benchmark is floating around. It's beating Google's best model, Carbon, by a fair margin and leads on 7 out of 8 parameters it was tested on. These are unverified and rumoured benchmarks, so take them with a pinch of salt.
5
1
24
2,215
holy crap is this true?? Better than Carbon? I thought this was going to be a mid tier model!!!
🚨 Gemini 4 Neon's benchmark is floating around. It's beating Google's best model, Carbon, by a fair margin and leads on 7 out of 8 parameters it was tested on. These are unverified and rumoured benchmarks, so take them with a pinch of salt.
1
1
2
79
Dhaval retweeted
An AI model with 816B parameters on a regular computer, even if you don't have hundreds of GB of RAM and VRAM! InferSSD has been created, an open-source project based on llama.cpp that allows running gigantic MoE models by reading their weights directly from an SSD. Instead of loading all experts into memory, the engine stores their weights on the disk and retrieves those that are needed during generation. The authors tested, among others: - Muse-Spark-Open: 816B parameters, 42B active. - DeepSeek-V4.1-Flash: 763B parameters, 8-16B active. And now the DeepSeek 763B results, which are impressive: - RTX 5090: 7.27 tok/s - DGX Spark: 7.26 tok/s - Apple M5 Max (48 GB): 2.87 tok/s - RTX 5060 Ti 16 GB: 2.55 tok/s The authors did not perform tests on the RTX 3090 24GB, but it should yield 3-5 tok/s On the RTX 5090, it is over 8 times faster than in llama.cpp with the same test configuration! The project supports macOS and Linux, uses standard GGUF files, and provides an OpenAI-compatible API. Is a few tokens per second an acceptable price for the ability to run a model with over 800B parameters locally? github.com/wici-ai/inferssd
16
13
2
115
10,819
RSI
Sundar Pichai: > The frontier will be developing the next set of models. > This is the era of RSI; this confirms Google has achieved it. > Confirms, internally, there is an inflection point after Argon. > It is tackling problems previous ones couldn't do. So those rumors were true; Google has achieved the RSI internally, and after the Argon release, there is lots of optimism internally. Google has passed the inflection point, and all upcoming models will be a different beast.
1
1
1
120
Dhaval retweeted
27 billion parameters in just 6.74 GB! And the mathematical results are almost identical to the full-precision model. A ttf was created, an open-source project that converts AI model weights into ternary values: -1, 0, and +1. The author compressed Qwen3.8-27B into two versions: - 8.20 GB: GSM8K 89.5%, identical to FP16. - 6.74 GB: GSM8K 88.0%. This is only about 1.6 bits per weight! Interestingly, the author did not limit themselves to quantization alone. After conversion, they conducted additional training to recover the lost quality. However, it did not come without compromises. In HumanEval, the larger version achieves 84.1%, while the base model reaches 93.3%. The model runs on a GPU with 12 GB of VRAM, but for now, it requires a modified llama.cpp. If such methods continue to be developed, running large models on cheap graphics cards may become significantly easier. Do ternary models have a chance to replace the 4-bit quantizations popular today?github.com/oguzeldereli/ttf-…
12
14
131
7,636
Dhaval retweeted
4-bit quantization did not worsen the AI agent's results, but it radically changed the way they solve problems! Researchers tested the Qwen2.5-Coder (7B, 14B, 32B) and Devstral 24B models in BF16, AWQ, and GPTQ versions on programming tasks. The results? No statistically significant drop in effectiveness was detected. But look at what happened to Qwen 7B after AWQ quantization: - First-attempt solutions: 17 -> 0 - Solutions requiring subsequent iterations: 10 -> 25 The model was still able to solve tasks, but instead of getting it right immediately, it had to correct its own code more often. This is particularly important for AI agents to which we entrust autonomous work. Two models may achieve a similar result in a benchmark while simultaneously performing a completely different number of operations, corrections, and tool calls. When testing quantized models, do you only check the final result, or also the path to achieving it? arxiv.org/abs/2611.02368
13
7
2
47
2,765
Dhaval retweeted
If you're buying a new phone to run AI models locally, pay attention to its storage type, not just RAM and CPU. Recent experiments have shown that even large LLMs can run on phones with limited RAM by streaming model weights directly from flash storage. That's where UFS 4.0 can make a huge difference compared to older UFS and eMMC storage. How fast is the storage in your phone? teapawt.org/swarogan/status/212206…
An 11-year-old iPad Air runs an AI model with 16 billion parameters. And that's on just 700 MB of available RAM! Someone created an open-source project called smolmaxxing, which, instead of keeping the entire model in RAM, reads the MoE weights directly from flash memory. Results from real devices: - 2013 iPad Air: Ling-mini-2.0 (16B) at 2.6–4.5 tok/s. - Android phone with about 3 GB of free RAM: Qwen3-Next-80B-A3B (80B) at 2.6–3.1 tok/s. The author declares that all results were measured directly on the devices. Of course, these are not record-breaking speeds, but running an 80B model on a phone with 3 GB of free RAM is impressive. On what oldest hardware have you managed to run a local LLM?
1
8
285
It happens with every company. Running a company is just like running a political party or a mafia syndicate. Cases will come and go, but business goes on as usual. Lawsuits are just part of the business.
🚨 OpenAI is badly messed up. California sues OpenAI over Hugging Face and other models for loss of control. Yesterday, they gave up their model's deep access to third-party labs to evaluate it, and that was a step in the direction of saving it from blowing up!
1
1
2
133
Dhaval retweeted
That's a good question. Given the pace OpenAI was moving at and everything we were hearing just a couple of months ago, I find it hard to believe safety is the real explanation. Their models score highly on alignment benchmarks, which only makes that explanation harder to buy. Tibo's in the basement.. Maybe something else is going on entirely. What if it's actually a massive breakthrough that's holding things up? With all the recent OpenAI news and the flood of posts about safety, it seems like they're taking it more seriously than ever. Maybe we're about to see an entirely new class of models. Or maybe they're genuinely facing major problems, though I find that hard to believe. It'll be fascinating to look back on this period and learn what was really happening behind the scenes.
OpenAI is scoring high on the alignment index in the @arena alignment index. GPT-6.1 Sol is the most aligned model we currently have. OpenAI has all its models in the top, and Opus 5.5 is sitting in 6th position. Now, I am not able to understand how Anthropic is releasing everything easily in the name of safety, and OpenAI is stuck in between. It's even beating Anthroic on fair margin on Labs alignment index as well. The question needs to asked, what really going on in the OpenAI, are they really supporting Anthropic for its public IPO on name of the no competitive model?
1
4
168
Dhaval retweeted
The video, that sold us dreams that OpenAI going to close the year on high. What a model it was, the best ever monster model they dropped GPT-6.0 Astra. After that it's just going down and down. I really wish to see OpenAI drop a model on par with Opus and Fable now. It's going to be really hard for them to beat the Anthropic at this pace, Anthropic is on roll and seems no one has answer to their pace now.
4
2
8
804
Dhaval retweeted
OpenAI is lagging behind Google might get ahead of OpenAI before the year ends
20
6
39
516
Dhaval retweeted
360 followers and I swear this is what it feels like every time I hit post 😭
🤖 Made with AI
2
1
6
240
Dhaval retweeted
Replying to @SiftByte @arena
@grok 什么叫对齐指数
1
1
1
193
Dhaval retweeted
This is what you call progress, I hope small models have such a big leap as well teapawt.org/arena/status/212215358…
Qwen4-Max by @Alibaba_Qwen has landed as #6 in the Code Arena: WebDev! With 1750 pts (preliminary), it's +74 pts over its predecessor, Qwen3.8-Max (0902) at 1676. It's also the top Chinese model in: - #6 HTML: 1748 pts - #8 Brand & Marketing: 1705 pts - #8 Reference-Based Design: 1770 pts Congrats to the @Alibaba_Qwen team on this release!
3
13
333