I am an #Immigrant & #Atheist. I am into #EV #renewables #renewableenergy. I can code a bit and local AI enthusiast
England, United Kingdom
Joined February 2009
- Tweets90K
- Following2.2K
- Followers2.6K
- Likes13.5K
RCO pipeline for MLX validated. GSQ is a no-go as it requires custom loader and patch to MLX-lm to work with shifted bits. Going to try some other ways of improving RCO quant
Hermit Dave retweeted
Everyone needs to know what actually happened in Serbia.
Instead of students just protest against their government, the students organized themselves into a student list and ran in elections against their government—and they have just won.
Dear Serbia, congratulations on your victory; you have just opened a new door in protesting. This is history
Downloading - no chance of managing SLQ for this - M3 Max 64GB is not capable of loading the BF16 model in memory
Will try and generate RTNs and then do a 4bit DWQ with 8bit teacher.
🚀Excited to announce Nanbeige4.3-33B!
To push the performance ceiling of small agents, our strategy in Nanbeige4.3 starts with building a powerful teacher first.
Nanbeige4.3-33B delivers SOTA agentic performance among open models at ~30B parameters:
* Strong results on agentic coding and cowork benchmarks.
* When integrated into mainstream agent harnesses, it tackles real-world tasks with cost-efficient inference.
* It will serve as a teacher for upcoming smaller Nanbeige models, starting with Nanbeige4.3-3B.
huggingface.co/Nanbeige/Nanb…
huggingface.co/Nanbeige/Nanb…
#LLM #Agent
Hermit Dave retweeted
Meghan Markle is getting more heat for being the daughter of an alleged pedophile than (Prince) Andrew got for actually being the alleged pedophile.
2nd SLQ MLX quant - Spark X2.6 4B for devices now available
huggingface.co/hermitdave/Sp…
Shall we try @IFM_AI's K2 Horizon 7B with Uno next ?
Started port of RCO to MLX. Rerunning SLQ for Spark 2.6 4B since 7:30 today - stage 2/4 was running for 5+ hours. Once that finishes I will run validation on RCO and continue with MoE quantisation pipeline
Definitely need M5U
Hermit Dave retweeted
Sad news today. After 5 years of medical training, one of my mates has been struck off due to a minor discretion. He slept with one of his patients and can no longer do the job he loves. What a waste of time and money. Genuinely a great guy and a brilliant vet
Kicked off SLQ port for MLX intial run yesterday. Started with smallest dense model I could find - Qwen3 1.7B model. SLQ process too a while - found whole lot of porting related bugs.
Defaulted to using heuristic / greedy allocation per layer
huggingface.co/hermitdave/Qw…
@ddalcu
I need a small / fast model with a long context window for quant related grunt work. Need to try SparkX2.6 4b - it has 1M context window. Need to try it out
Thanks for posting about SLQ.
I kicked off SLQ-MLX port earlier and just finished validation of a 4.5bpw quant
Will create a few quants including Qwen3.8 27B and LFM2.5 24B in the next few hours
Introducing statistically-lossless Qwen3.8-27B!
Three models, 23.7–30.5 GB: up to 57% smaller weights than BF16.
98–99% measured EAR, versus 97.71% for native FP8.
EAR measures overlap with BF16’s next-token probabilities. A 99% EAR means 99% average probability overlap, here, measured over the reference’s top-10 tokens after renormalization.
The search optimizes EAR directly, assigning more precision to sensitive projections while keeping activations in BF16. The goal is to find the smallest checkpoint while keeping fidelity as high as possible.
vLLM + Humming • Apache 2.0
GGUF versions coming next week!
Get the models:
huggingface.co/collections/k…
Based on SLQ (arxiv.org/abs/2605.02404) from ISTA’s DASLab (github.com/IST-DASLab), the group behind GPTQ, GSQ and RCO, with customizations to improve the balance between size and fidelity.
More details soon.
A new 40B A2.5B MoE model.. 4 bit MLX quant 22GB. huggingface.co/RecursiveSI/f…
Going to try it out.
My 2/3/8 bit quant with working MTP now on HuggingFace @ornith_
huggingface.co/hermitdave/Or…
Working on M3 64GB Mac with 38-45 TPS
Aloha!🌺 Meet Ornith-1.5-122B: No Tradeoff Between Strong and Fast
Stronger. Faster. Leaner. Ornith-1.5-122B delivers frontier-level capabilities with exceptional inference efficiency. With superior instruction following, stronger task decomposition, and deeper understanding of real-world workflows, it achieves state-of-the-art performance among open-source models of comparable size, while running on a single NVIDIA DGX card!!
Open weights under the MIT license.
Coding
✅SWE-bench Verified: 0.845
✅TB2.1: 0.813
✅CyberGym: 0.800
✅NL2Repo: 0.512
Agentic
✅HLE: 0.446
✅BrowseComp: 0.785
✅MCP-Atlas: 0.746
✅Toolathlon: 0.708
🤗Hugging Face: huggingface.co/ornith-ai/Orn…
We can’t wait to see what you build with Ornith-1.5-122B.
Hermit Dave retweeted
Aloha!🌺 Meet Ornith-1.5-122B: No Tradeoff Between Strong and Fast
Stronger. Faster. Leaner. Ornith-1.5-122B delivers frontier-level capabilities with exceptional inference efficiency. With superior instruction following, stronger task decomposition, and deeper understanding of real-world workflows, it achieves state-of-the-art performance among open-source models of comparable size, while running on a single NVIDIA DGX card!!
Open weights under the MIT license.
Coding
✅SWE-bench Verified: 0.845
✅TB2.1: 0.813
✅CyberGym: 0.800
✅NL2Repo: 0.512
Agentic
✅HLE: 0.446
✅BrowseComp: 0.785
✅MCP-Atlas: 0.746
✅Toolathlon: 0.708
🤗Hugging Face: huggingface.co/ornith-ai/Orn…
We can’t wait to see what you build with Ornith-1.5-122B.
