大模型现在的测试确实相当麻烦
Alibaba engineers found a nasty quantization bug while running DeepSeek-V4.1 on T-Head Zhenwu 810E chips.
The problem was with DeepSeek's Engram weights. They're supposed to stay in BF16 because their values are very small.
But vLLM renamed these weights internally, so its ignore list couldn't recognize them. As a result, it silently converted them to 8-bit integers, rounding most of their values down to zero.
Basically, the model was running with corrupted weights without throwing an error. wtf.
1
34
fatlj retweeted
After an overnight run, I now have the DeepSeek-V4.1-Flash-1017 GSQ-RCO Coder IQ2_XXS (ISTA-DASLab) results from the Elixir SWE benchmark, and this might be the most interesting finding this benchmark has produced so far🤔
DeepSeek is massively hyped. I was expecting a score of around 15–16 out of 20, but it scored just 12 - only one point more than Qwen 3.8 Next Flash Gyro-L and only two points more than the smallest Qwen 3.8 Next Flash, which can run on a single 3090 + RAM!!!
On top of that, DeepSeek required 5x3090s + RAM and generated tokens at just 40–50 tokens/sec, while the Qwen models nearly doubled that speed.
Choose your models wisely. There's a LOT of hype out there, but the actual results don't always tell the same story!
好像也不是完全没用
decode 的速度确实会影响使用体验,可任务完成速度取决于端到端 tps(用户输入到结束), prefill 速度影响也大。同时 coding agent 本身就是 input 远大于 output
gpt-6.1-sol 最近的优化就是明显的反例,decode 快了,首字巨慢,解决问题巨慢。
给 4090 D 解锁 Tensor 算力后,有朋友提醒:“这对跑 MoE 量化推理可能几乎没用。”
测完了,确实如此:基本零收益😭
翻了 Strata 源码,IQ3_S 专家内核实际跑的是纯整数指令(mma s8.s8.s32),解锁翻倍的只是浮点 Tensor 算力,根本使不上劲;而且异构推理瓶颈全卡在显存带宽、PCIe 搬运和 CPU 卸载上。
受控对照数据与汇编分析写了篇总结: blog.ferstar.org/posts/tenso…
36
呵呵,到中国几天?用了中国 app 几天?使用英文体验中文 app?
As much as I respect Chinese-made AI models and hardware, I'm equally puzzled by how rough most of the software still is. Whether it's Wechat or Alipay or HarmonyOS. You'd think getting unique aesthetics, perfect fidelity, and great flows would be "easy", but clearly it is not.
1
11










