nihui (@nihui) header
nihui (@nihui) profile picture

春雨为观,香月为邻 | 喜欢玩游戏时看风景 | 想成为优秀的程序员 | 看上去是女装大佬实际并不是( | 上海老阿姨 | qwqwqwq

Shanghai
Joined July 2008
Photo by nihui (@nihui)Photo by nihui (@nihui)Photo by nihui (@nihui)Photo by nihui (@nihui)Photo by nihui (@nihui)Photo by nihui (@nihui)Photo by nihui (@nihui)Photo by nihui (@nihui)Photo by nihui (@nihui)Photo by nihui (@nihui)
nihui retweeted
三个月后...它从 13 t/s -> 128 t/s 它进化了,我也成长了😋
折腾到大半夜的,汇报下这几天深耕 ncnn-MoE-Runtime 的进展! 好消息是,它可以在 5070ti (16G vRAM)+ 9800X3D + 32G (7600Mhz) 内存组合下情况下,可以跑起 GPT-OSS-120B 的模型 且最高峰值可达 13.36 token/s 想到将来能够通过更低的硬件成本,能够运行起更多的 LLM 时就非常的兴奋! 接下来继续优化,让我来探索下极限边界在哪! 项目地址:github.com/crafcat7/ncnn-MoE… 蹲点大佬们的 star(目前优化还没上传!目前仓库内仅能运行起 120b 模型)
1
7
1,056
This screenshot shows Qwen3-0.6B LLM running on a pure IA32 processor such as the Intel Core Solo T1350, based on github.com/crafcat7/ncnn-MoE… from @crafcat7.
2
2
1
7
941
I reverse engineered ARKit to learn how it works and bust the myths. It is a ML model that takes a 128x128 pixel RGB selfie and spits out the blend shapes, plus post-processing. I made it work on macOS, with any webcam. OpenVT mac builds coming soon. gist.github.com/hoshinolina/…
6
37
4
301
8,642
libwaifu/waifu2d 第一版完成!🎉 虽然头发和口型还有些问题,但至少waifu能动了!目前整套流程都是自动化的:AI 生成图片→See-through模型分层→驱动动画,无需任何手工修改。感谢See-through项目:github.com/shitagaki-lab/see… 如果感兴趣,欢迎给 libwaifu 点个 ⭐ github.com/ling0322/libwaifu
21
113
4
1,574
52,113
Qwen-Image-2.1 running on Intel B580! Thanks to @nihui for qwenimage-ncnn-vulkan
1
12
1,242
nihui retweeted
Visualizing to learn model internals in the agentic era can be surprisingly low-friction. I had Codex one-shot architecture notes earlier for several recent OSS models—DeepSeek V4/V4.1, GLM 5.3, and Kimi K3—specifically from a kernel-development perspective: tensor contracts, exact shapes/layouts, attention variants, KV-cache representation, precision choices, dataflow, and operator boundaries. Sharing in case they’re useful to others: lei.chat/models/
5
19
1
117
10,532
好耶,libwaifu支持Qwen-Image-2.1成功🎉 github.com/ling0322/libwaifu
1
1
25
2,661
I wanted to add extra GPU options to the benchmark suite, sooo I added @nihui's vkpeak to cover a range of options. Only the @RadxaComputer X4L on sbc.compare/vkpeak so far, but as with others, new boards will be tested with it, and others backfilled later! For full data, the table at the bottom of the comparison pages such as sbc.compare/radxa-x4l is where it will live.
2
1
6
801
Vulkan `vkpeak` benchmark for the new Arduino Ventuno Q reaches 400+ FP32 GFLOPS. ./vkpeak 0 device = Adreno623 driver = turnip Mesa driver / Mesa 25.2.8-0ubuntu0.24.04.2 fp32-scalar = 417.44 GFLOPS fp32-vec4 = 438.42 GFLOPS fp16-scalar = 594.99 GFLOPS fp16-vec4 = 853.58 GFLOPS int32-scalar = 109.97 GIOPS int32-vec4 = 111.09 GIOPS int16-scalar = 283.50 GIOPS int16-vec4 = 294.14 GIOPS int64-scalar = 11.31 GIOPS int64-vec4 = 6.33 GIOPS int8-dotprod = 621.29 GIOPS copy-h2h = 10.11 GBPS copy-h2d = 11.41 GBPS copy-d2h = 9.12 GBPS copy-d2d = 13.66 GBPS @arduino
1
1
4
867
nihui retweeted
你最牛了!
471
210
120
4,020
1,402,514
nihui retweeted
YuzukiNeko - A Pico Shape Cute CatBoard, With Allwinner F101 SoC Hardware OpenSource under CC0: github.com/YuzukiHD/YuzukiNe… Linux Kernel: github.com/YuzukiHD/linux-ma… Buildroot: github.com/YuzukiHD/buildroo… Zephyr: github.com/YuzukiHD/zephyr
8
42
5
438
18,337
From dense models to 122B and 295B MoE, local inference should work with the hardware you have—not require a datacenter. Palm-Infra is open source. Built by Tencent's #PalmAI team, it brings Metal, CPU SIMD and SSD expert offload to Apple Silicon.
3
5
3
16
20,628
11k views in 1 day for a technical 1h 40min video! My hope for humanity has been restored. There's still people out there watching long videos :) youtube.com/watch?v=aQv9pUl9…
23
36
820
21,176
nihui retweeted
太好了,上周做的 DSV4 flash 的适配可以直接用在 DeepSeek V4 Flash 0731 的版本上 5070ti 跑 DeepSeek V4 flash )
3
1
10
3,283
Two failed attempts. One working solution. Building a real-time forklift proximity alert system on a Raspberry Pi 5 taught me the biggest bottleneck wasn't the model, it was the deployment stack. What ONNX Runtime, INT8 quantization and NCNN taught me: medium.com/p/two-failed-atte…
3
1
4
1,444
nihui retweeted
先让 ncnn 能运行起 GPT-oss-20b 然后再一步步完善补充 ncnn MoE Runtime 的伟业!Make ncnn Great Again!
1
5
1,778
第一次参与线下的公开演讲,总之就是非常激动!🤩
66
4
116
9,357
轨道交通享受者
139
58
3
2,642
77,686
Didn't expect the PoC prototype worked! I just built a zero runtime dependencies CNN inference server using #golang by wrapping #ncnn (Nihui Convolutional Neural Network) into #wazero WebAssembly runtime, and it cross compile without needing CGO support 🔥
1
1
3
1,637