Crouching VRAM, Sleeping Dragon: How the Strata Family Makes Massive AI Models Run by Waking Only What They Need
uncategorized.blog/posts/cro…
#strata #glm #localLLM #inference #nvidia #rtx5090
OffByQuant retweeted
The Quantization Tax Nobody Charged: #Qwen 3.8 27B Q4 Quant Matches Its Published Scores on a Single #RTX5090 @Alibaba_Qwen
youtu.be/qhGds3DVizg
uncategorized.blog/posts/the…
#localLLM
Everyone budgets a quality tax for 4-bit quants. I went looking for it on #Qwen 3.8 27B: GPQA, IFBench, 95 real CVEs, RULER at 262K, KL-divergence.
The tax never showed up — Q4 matches @Alibaba_Qwen 's published BF16 scores on a single #RTX5090.
uncategorized.blog/posts/the…
#localLLM
OffByQuant retweeted
PyDiffWatch: watches the PyPI release firehose, diffs every new version, flags suspicious ones fast enough to report for takedown.
▸never runs what it inspects – packages are data, not code
▸reviews run on a local LLM
github.com/OffByQuant/PyDiff…
@ramimacisabird @abh1sek @anantshri
