who in the @AMD @AIatAMD @AMDRadeon community are the MUST follows? i know @Italianclownz and @1337hero but who else....whos going to show me the ways of the AMD GPU once this r9700 gets here?

WELP - A Reproducible Evaluation Lifecycle for Local AI because Most local model testing ends with a few tokens-per-second numbers, a benchmark table, or “this feels good.” That wasn’t enough for me. WELP - the WumboLabs Evaluation Lifecycle Protocol, is my attempt to make local model evaluation reproducible, phase-gated, and honest about what the evidence actually supports. read more here: wumbocore.com/labs/ i will be putting a more detailed write up as an article soon.

i have been seeing a whole lot more of this one the timeline lately, and its beautiful.

I have mostly wrapped up all the testing with qwen3.8-27b for now. found some interesting things Unsloth Qwen3.8-27B UD-Q2_K_XL is an unusually capable 27B-class model for this RTX 5070 12GB configuration, especially for reviewed coding and bounded reasoning. The final reliability campaign nevertheless found too many hallucinations and strict-interface failures for unguarded daily use. full report can be found here: wumbocore.com/records/qwen38…

Qwen3.8-27B showdown on my RTX 5070 12GB using LLMGauge and llama.cpp. The goal was not just to find which GGUF could technically load. I wanted the best daily model on this hardware after accounting for quality, coding, reliability, hallucinations, context, VRAM, speed, reasoning, KV cache, sampling, and speculative decoding. WINNER @UnslothAI Qwen3.8-27B UD-IQ2_M Weighted scores • Unsloth UD-IQ2_M: 0.9264 • XYZ Q3: 0.7583 • AtomicChat AD-IQ2_XXS: 0.6606 The weighting favored bounded quality, coding correctness, and reliability over raw throughput. AtomicChat can win narrower speed/headroom comparisons, so the ranking would change if speed were weighted much more heavily. Bounded evidence index • Unsloth: 22/24 • XYZ: 19/24 • AtomicChat: 16/24 That index is only for this study, hardware, runtime, and prompt set. It is not a universal model score. CODING AND RELIABILITY Reviewed coding tasks • Unsloth: 10/11 • XYZ: 8/11 • AtomicChat: 7/11 Strict three-seed contract reliability • Unsloth: 3/3 • XYZ: 2/3 • AtomicChat: 0/3 All three finalists completed 50/50 primary 8K Q4/Q4 prompts. Completing a benchmark is not the same as producing good answers. Manual review still exposed instruction-following, evidence, coding, and reliability differences. DAILY DRIVER CONFIG WumboJetsII Model: Unsloth Qwen3.8-27B UD-IQ2_M Runtime: llama.cpp b10449 (commit 0d9ceae1e) GPU: RTX 5070 12GB CUDA0 only all layers on GPU parallel=1 fit off KV offload enabled Flash Attention enabled Context: 16,384 Batch: 256 UBatch: 64 KV cache: Q4_0 K / Q4_0 V Reasoning: off by default Preserve thinking: off Speculation: none by default Sampling • temperature 0.7 • top-p 0.8 • top-k 20 Maximum normal output: 2,048 tokens The final representative 16K rerun completed and validated 14/14 prompts. The checked generic and coding contracts passed. Minimum observed headroom in that matrix: ~1,637 MiB Separate fully filled 16K retrieval run: ~1,608 MiB retained That is a big reason I prefer this over the largest quant that will technically initialize. It leaves a real operating margin instead of living against the VRAM wall. CONTEXT SCALING I tested context instead of trusting the advertised window. Unsloth UD-IQ2_XXS completed exact fully filled beginning/middle/end retrieval at: 4K / 8K / 16K / 32K / 64K / 96K At 96K: • ~616.6 prompt tok/s • ~27.7 generation tok/s 128K failed during context initialization. CUDA could not allocate a 538.27 MiB compute buffer. On this 12GB card: • 96K is the highest verified filled-retrieval edge • it is not what I would recommend as the normal daily config Two useful Unsloth profiles • UD-IQ2_M at 16K as the primary daily driver • UD-IQ2_XXS as the large-context specialist when max context matters more than quality KV CACHE Higher-precision KV vs Q4_0/Q4_0 showed no observed retrieval benefit that justified the extra VRAM. Q4_0/Q4_0 preserved exact 16K retrieval across all finalists and gave the most useful headroom. XYZ Q3 OOMed at 16K using F16/F16 KV. On a 12GB card, Q4/Q4 KV was not just a way to squeeze in more context. In these tests it was the more practical setup with no observed retrieval penalty. REASONING I tested the community suggestion that Qwen3.8 works well with medium reasoning and a 16K thinking budget. Mixed results: • Medium reasoning ran with the requested 16K thinking budget and retained ~1,270 MiB headroom, but on the evaluated strict task it generated a forbidden second code block • High reasoning produced the requested state-transition response correctly, but took 150.3 seconds Conclusion • Reasoning works and can be useful • I would not enable it globally • Daily config keeps reasoning off and enables it only for tasks that actually benefit Preserve-thinking ON did not improve the reviewed two-turn task. Preserve OFF was shorter and safer. Reasoning history preservation stays disabled by default. SPECULATIVE DECODING / MTP At 8K, native MTP increased generation throughput by roughly 1.57x to 2.35x. Mean wall time dropped to about 61.1% of the non-MTP baseline. Cost: ~0.7 GiB extra VRAM Worst observed headroom: ~1,090 MiB I am not using MTP in the normal 16K daily config. An 8K performance profile with MTP enabled looks genuinely useful. PACKAGE COMPARISON Unsloth UD-IQ2_XXS • Full GPU, 66/66 layers • ~7,974 MiB CUDA model buffer • ~2,539 MiB free at 4K • Exact admission passed Unsloth UD-IQ2_M • Full GPU, 66/66 layers • ~9,228 MiB CUDA model buffer • ~1,351 MiB free at 4K • Exact admission passed AtomicChat AD-IQ2_XXS • Full GPU, 66/66 layers • ~7,628 MiB CUDA model buffer • ~3,014 MiB free at 4K • Exact admission passed AtomicChat AD-IQ1_M • Full GPU, 66/66 layers • ~7,171 MiB CUDA model buffer • ~3,241 MiB free at 4K • Exact admission passed XYZ Q2 • Full GPU, 66/66 layers • ~7,544 MiB CUDA model buffer • ~3,035 MiB free at 4K • Exact admission passed XYZ Q3 • Full GPU, 66/66 layers • ~9,721 MiB CUDA model buffer • only ~857 MiB free at 4K • Exact admission passed, but already below a comfortable guarded margin XYZ Q1Q • Loaded 65/65 layers • ~5,608 MiB CUDA model buffer • ~5,037 MiB free • Immediately produced EOG instead of the requested exact answer XYZ Q1Z • All 66 layers loaded • Generated ADMISSION_OK at ~50.83 tok/s • Failed to terminate and had to be killed at the 180-second timeout The lowest-memory artifact was definitely not automatically the most usable artifact. Empero Qwen3.8-27B Ridge 3.7bpw Interesting architecture-aware quant, but it does not fit under the rules I set. It failed strict full-GPU admission during model-weight allocation, before context or KV allocation. No CPU fallback or hybrid execution was allowed. Ridge is a physical non-fit for this 12GB full-GPU configuration, so I did not pretend to compare quality against it. Unsloth Q4_K_M was also physically too large for the full-GPU requirement. TESTING RULES Intentionally strict: • one RTX 5070 • 12,227 MiB physical VRAM • 250W stock power limit • one active request • parallel=1 • fit off • all model layers on GPU • GPU KV • GPU recurrent state • no CPU model offload • no CPU KV • no hybrid execution I wanted results for what this card can actually do as a full-GPU local inference device, not what can be made to technically work by spilling pieces onto system RAM. LLMGAUGE IMPROVEMENTS The study originally hit a blocker because LLMGauge could not fully represent several llama.cpp controls I needed. That work is now implemented. First-class support for: • fit control • reasoning preservation • speculative type • top-k • seed • independent K/V cache types • KV offload • parallel=1 • reasoning effort • reasoning budget • large-prompt file transport • durable effective runtime-command evidence Result artifacts now distinguish an omitted runtime default from an explicitly requested setting. Fit can explicitly record on/off. Reasoning preservation records true/false. Speculation records a canonical selection, with explicit none distinguished from omission. Invalid or contradictory speculative settings fail closed. The changes propagate through runtime evidence, result validation, reporting, comparison, fingerprints, and sanitized public export. A live validation run also uncovered a public-export bug involving native execution evidence and JSON-pointer path redaction. That was fixed rather than worked around. Another live sweep exposed a backward-compatibility issue with preserved v3 fingerprints. That was also fixed. New controlled runs moved to fingerprint v4 while historical artifacts remained verifiable. Final validation • 994 tests passed • 1 skipped • Ruff passed • git diff --check passed • 68/68 retained showdown result directories validated • 20/20 required machine-readable study summaries parsed correctly • three final 16K winner runs independently passed LLMGauge validate-result • live fit-off, preserve-on, preserve-off, and draft-MTP control smokes passed • final sanitized public export validated and did not mutate its private source tree • no leftover llama.cpp, SGLang, or vLLM processes after the run SYSTEM STABILITY Earlier in the work, the RTX 5070 hit an NVIDIA Xid 109 CTX SWITCH TIMEOUT. That put the GPU into RESET_REQUIRED and ultimately took down Hyprland because the same GPU was driving the desktop. There was no active model benchmark at the fatal timestamp, so I am not attributing the Xid to Qwen3.8 or any particular quant. For the resumed study I ran one GPU request at a time and continuously monitored GPU telemetry and kernel/NVIDIA events for 1 hour 58 minutes. No new Xid 109 or reset-required state occurred. Final GPU state after the completed evaluation: • 542 MiB used • 11,269 MiB free • 41 C • P8 • 9.63 W The original Xid remains unattributed rather than being retroactively labeled as a model failure. LIMITATIONS Manual review covered all Coding Core responses, deterministic Generic Core boundaries, practical prompt excerpts, repeated seeds, reasoning comparisons, preserve-thinking tests, and MTP pairs. It did not assign exhaustive human rubric scores to every prose answer. Generated model code was inspected and parsed, but never executed. Requested runtime flags prove what was requested from llama.cpp. They do not by themselves prove that every model/template internally complied exactly as intended. The current structured LLMGauge artifact also does not expose explicit split-mode even though these tests used one physical visible GPU and the direct llama.cpp admission work used split-mode none. The recommended sampling profile is only a partial reproduction of the producer profile because LLMGauge does not currently expose the recommended presence penalty. These results apply to these exact models, builds, prompts, runtime versions, and this RTX 5070. FINAL PRACTICAL RESULT Best overall: Unsloth Qwen3.8-27B UD-IQ2_M Best coding: Unsloth UD-IQ2_M Best reliability: Unsloth UD-IQ2_M Best large-context option: Unsloth UD-IQ2_XXS Best narrower speed/headroom alternative: AtomicChat AD-IQ2_XXS Interesting runner-up: XYZ Q3 Ridge 3.7bpw: does not fit strict full-GPU on 12GB Daily configuration • UD-IQ2_M • 16K context • Q4_0/Q4_0 KV • Flash Attention • parallel=1 • fit off • reasoning off • preserve thinking off • MTP off • temperature 0.7 • top-p 0.8 • top-k 20 Optional performance profile: 8K + native MTP Optional difficult-task profile: enable reasoning selectively Optional large-context profile: UD-IQ2_XXS, with 96K as the highest fully filled retrieval config I verified TAKEAWAY The “best model that fits” is not necessarily the largest quant, the fastest quant, or the model with the most advertised context. On a 12GB card, the best result came from balancing model quality, coding reliability, KV precision, context, VRAM headroom, reasoning behavior, and runtime configuration together. For my RTX 5070, that balance currently lands on Qwen3.8-27B UD-IQ2_M.

i am happy to annouce that after far to long. LLMGauge now has an option to export and submit directly to localmaxxing.com by the great @LottoLabs here is a @atomic_chat_hq Qwen3.8-27b run with my 5070: localmaxxing.com/en/runs/cms… check out LLMGauge here: github.com/wumbolabs/llmgaug…

phase 1 underway: here is the first public release of wumbo-quickshell. a desktop shell for Linux users running Wayland and @hyprwm. this all started after seeing some of the stuff that @dhh was doing with quickshell on omarchy and it got the gears turning in a direction i had been thinking about for a while...a LLM/Agentic focused Linux Distribution. and with that i building an "agent status" module to see when my agent was done working without having to watch the terminal and has slowly evolved from there. this is only the beginning, i have already started working on wumbosd; the daemon that will bring in the attention center and persistent, remote connectivity to your agents directly on your desktop and more. stay tuned for more. I have attached a few screen shots of the bar modules in their current v1.0 state. Agent Status System Monitor Audio Control Panel Date/time/Weather. github.com/WumboLabs/wumbo-q…

I tested 3 Gemma 4 12B variants on an RTX 5070 12GB. The surprise: The heavier Q5 quant lost. Winner: Gemma 4 12B IT QAT UD-Q4_K_XL • 4.32/5 practical-use score • 73.5 tok/s • 7.5GB VRAM used • ~4.7GB headroom It beat UD-Q5 not just on efficiency—but on practical usefulness. The Q5 model used ~1.8GB more VRAM, ran ~14 tok/s slower, and didn't produce better answers. LLMGauge tested: • Linux/NVIDIA guidance • Python coding • Docker review • Honesty/safety • Summarization • Local LLM advice Real hardware. Real testing. No hype. Full report: wumbocore.com/benchmarks/gem…

I tested JetBrains Mellum2 12B-A2.5B Instruct Q4_K_M with LLMGauge v0.20 on real local hardware: llama.cpp CUDA on an NVIDIA RTX 5070 12GB. The big result: Mellum2 Instruct is a strong local 64k agent-backend candidate. It passed the 8k fake-tool smoke test, the full 8k agent-backend suite, the 8k/16k/32k context ladder, the 64k fake-tool smoke test, the 64k synthetic-agent-preload test, and finally the full 64k agent-backend suite. At 64k context, the full suite completed 5/5 prompts with valid artifacts, no failures, peak VRAM around 9203 MiB, roughly 3024 MiB of VRAM headroom, and generation speed in the 251–257 tok/s range. That is a very strong fit/performance result for a 12GB GPU. Quality-wise, Mellum2 Instruct looks fast, stable, and generally conservative. It showed good fake-tool resistance, handled coding tasks cleanly, and stayed safe in long-context agent-style prompts. The main caution is shell/systemd safety: it sometimes moves too quickly toward actions like enabling or removing service files, so that area needs stricter manual scoring before trusting it for real ops-agent workflows. Verdict for Mellum2 Instruct: KEEP. It earns a spot as a fast 64k local agent-backend candidate, with safety caveats around shell/systemd recovery behavior. Next, I'll test the sister model, Mellum2 Thinking, using the same controlled LLMGauge process. We’ll start with a small 8k fake-tool honesty smoke test, then move to the full 8k agent-backend suite, then an 8k/16k/32k ladder, and only then try 64k tests if VRAM headroom and stability look good. The main question for Mellum2 Thinking will be whether its reasoning-oriented behavior improves planning, debugging, and long-context task handling without becoming slower, more verbose, or less conservative than the Instruct model.

Monolith is public now as a technical alpha. It’s my local-first AI workbench for testing GGUF models, llama.cpp profiles, prompt suites, context scaling, downloads, and practical local model eval. Still early. Manual setup. Built for real local testing, not hype. github.com/WumboLabs/monolit…