CuteAFD EXLR8 update… v4 is close, I can smell it, and it brings EXLR8 support for GLM 5.3, atleast some… I didn’t try every bitrate just up1 😊 It has push loading for 40GB/s speed loading with 6 sparks 🚀 (necessary so I can let my agents run lots of dev cycles) Mostly I spent lots of cycles on kernel duels beyond the basic wiring so Gemini/Astra/DeepSeek battled it out (with a bit of sol 6.1 and opus too). In the end I also ran duels for about 50% of the total time spent on EXLR8 duels for EXL3 mcg. I use real routing traces for the experts now for the simulated duels because reality of the engine stopped lining up with synthetic profiles. I didn’t do that yet for other families just GLM to do a real assessment of performance. I also had it smash the decode kernels too because lots of the deep core is compressed in EXLR8 (my quant is fp8 for now, will have a 6bpw EXL3 variant tomorrow to measure). The quality level of the EXLR8 quants is exceptional! The choices for the deep core made it beat out my quant hands down (0.02 vs 0.03 KL, Top-1 within 0.1%). I was surprised and realized I made my quant from the FP8 core 🤦 so I am remedying that. I expected my process to produce slightly higher results because it does only one bpw and feeds the activations and errors up through the layers so the layers are somewhat calibrated with each other. My new quant won’t be apples to apples either though because i used straight EXL3 6 bit for all the parts I am compressing… this has quite a bit more precision than fp8 so I think my new quant will win. On the decode side, I’m not sure if I optimized enough on EXLR8, but 56tps is pretty good. For EXL3, I got 63 but the core isn’t compressed… so I guess that may come down a bit with the new quant. For prefill, I thought I had solved the slowness of EXLR8 vs EXL3 for prefill, but it turns out I had regressed my EXL3 batching and once I fixed that, EXL3 was handily ahead 3500 vs 2800 tps. I’m not totally sure why, maybe rounding adds just enough extra work that the duels couldn’t get quite as much efficiency in the weird language of NV compute space. Possibly will share some nsight analysis after the release, GPUs busy checking many things right now. From a performance perspective, A8 on prefill can get another 20% but it is a quality tradeoff. Maybe it’s fine for prompt processing but these measurements are doing A16 prefill. Decode is always A16, no reason to lose quality there… because the compute isn’t the limit A8 doesn’t speed up. A8 EXRL8 would be just a hair slower 3350 vs 3500 EXL3 A16… I’ll post a run later, I’m estimating the boost from an earlier build. EXLR8 looks like it scales a little bit better on the concurrent decode. c16 152 v 146, +4% I guess the end result means I will use standard EXL3 mcg as my daily driver. I will have one revisit on performance w.r.t decode/KL once my new quant is ready. My setup is wildly different since I have tons of VRAM, don’t need to dynamically adjust quality, and I’m doing AFD which alters timings and tosses an RTX in the mix. On all sparks I guess EXLR8 would dominate my quant because of the memory bandwidth for the dense parts, and that probably still is the case for the next quant. This was fun! Found several bugs, and practice my GPU kernel engineering management skills 😂

this is pretty evil and clever. so if your samsung TV model doesnt have a microphone, it will look for connected devices with one, such as a soundbar or a connected bluetooth device. this gets shipped off to bixby.

NEW: Trump called Musk and asked him to lift restrictions on a military-grade version of Starlink in Ukraine that the SpaceX chief had throttled, U.S. officials said. The system was working more normally Thursday, shortly after their conversation, per Us and Ukrainian officials

Oh god: The p384 private keys are randomly selected integers between 1 and 256, the private keys are effectively all-zeroes apart from the last byte

Business Insider also reporting that Amazon projects it would take delivery of $3 billion a month in GPUs “every month for the foreseeable future,” all while not having anywhere to put them, as part of an “early rack receipt” program. Extremely strange!