Chief Scientist, AI4AI Lab
Joined January 2023
- Tweets172
- Following32
- Followers12.7K
- Likes32.3K
The ForgeKernel paper is now released. Last week, we boosted DeepSeek V4.1 FlashMLA real-time by 9.8% using ForgeKernel.
Paper: arxiv.org/abs/2611.04998
Check out what the paper includes:
1. Architectural decision-making: We propose an agent architecture that separates architectural design from code implementation, guides implementation through explicit architectural decisions, and revises these decisions based on performance feedback. When optimisation encounters a structural bottleneck, the method revisits work partitioning, dataflow, or pipeline organisation rather than only tuning locally within a fixed architecture. Across runs, it also re-assesses an existing implementation's architecture rather than directly carrying it forward.
2. Measurement-validated mechanism for cross-run self-evolution: Hardware mechanisms discovered during our kernel optimisation are validated through independent probes on the target hardware before being stored in a knowledge base shared across operators and input shapes. The knowledge base guides architectural design and implementation planning in subsequent runs, converting findings from individual runs into reusable knowledge.
3. Introducing ForgeKernel: It is a self-evolving operator optimisation system that integrates architectural revision, reference updates, and knowledge reuse. Evaluation on NVIDIA Hopper and Blackwell GPUs and Huawei Ascend 910C accelerator demonstrates performance gains over expert and generated references, including a 1.10× geometric-mean speedup on NVIDIA GPUs. As of 2026-10-30, submissions generated by ForgeKernel rank first on all four leaderboards: the three CANN-Bench 2.0 solution leaderboards (Ascend 910C, 950DT, and 950PR) and the SOL-ExecBench FlashInfer-Bench track.
ForgeKernel Live ended with a 9.8% increase in the efficiency of DeepSeek V4.1-FlashMLA Prefill. We decided to share our excitement with everyone by open-sourcing ForgeKernel.
ForgeKernel Live page:
openbmb.cn/forgekernel/
Open-source link: github.com/OpenBMB/ForgeKern…
Quick catch-up on ForgeKernel Live:
⌛️The RSI kernel optimisation had been running for nearly a WHOLE day;
✅31 versions of DeepSeek V4.1-FlashMLA Prefill are listed on the page, written from scratch in an RSI CLI black box without seeing official code;
📣It first beat the FlashMLA baseline in Version 13 within 10 hours, and reached 109.8% efficiency in the 23rd hour.
After all these end, we are excited to open-source:
1. DeepSeek V4.1 FlashMLA Prefill optimised version: What we have improved in ForgeKernel Live, representing the frontier of human expert performance in kernel optimisation.
2. All kernels from experiments in the ForgeKernel paper (coming soon): Covering the strongest human expert implementations — DeepSeek V3/V4, Flash Attention 4, Qwen GDN, Kimi Delta Attention — as well as the best kernels released by Kernel Agents such as CAKE and KDA. The kernels generated in our ForgeKernel experiments surpass all of the above.
3. The complete ForgeKernel Harness: You may use it to optimise any kernel.
We are displaying real-time RSI kernel optimisation.
See this site: openbmb.cn/forgekernel/ teapawt.org/i/broadcasts/2brnWkXLO…
We are displaying real-time RSI kernel optimisation.
See this site: openbmb.cn/forgekernel/ teapawt.org/i/broadcasts/2brnWkXLO…
Based on our previous discovery on JustGoal, we redesigned how we do kernel optimisation and launched a self-evolving kernel optimisation harness called ForgeKernel, a part of our Forge series.
We tried ForgeKernel on SOL-ExecBench(FlashInfer-Bench) from NVIDIA and CANN-Bench from HUAWEI. It performed well and easily reached #1 in every list we tested.
Here are the ranks:
research.nvidia.com/benchmar…
cannbench.com/leaderboard
We'll release another paper on ForgeKernel soon.
Do we still need specialized agents for GPU kernel optimization?
We asked the question, and ran the experiment to answer it.
New paper with @THU_NLP: Agents under Goal mode are now able to beat systems deliberately built for kernel optimization. We named it JustGoal.
On FlashAttention, Gated DeltaNet, and DeepSeek Sparse Attention, a general frontier model (in our case, using Fable 5.1) delivers 1.08× to 1.58× speedup over expert-written references. Against the specialized KDA-0.5 (1.45× and 1.28×), Fable 5.1 hits 1.58× on both.
Frontier models are really burying the talent of kernel optimization experts in the past.
Paper: github.com/OpenBMB/JustGoal-…
At the frontier, the model capacity has caught up in optimising kernels.
The full paper and code will be available very soon. And huge props to the team — a creative collaboration between @THU_NLP and ModelBest's AI for AI Lab.
Sometimes the simplest questions lead to the most interesting answers.
1
87
Honestly, the results surprised us too.
Fable 5.1 performs the mission at or above on all three — 1.08× to 1.58× over the expert references. KDA-0.5, which is built specifically for this job, scores 1.45× and 1.28× on the same operators, while Fable 5.1, carrying zero kernel-specific machinery, hits 1.58× on both.
This means the general agents easily passed the best implementation within hours.
1
112
The default answer of kernel optimisation has been "specialised": each complex system wraps an LLM in elaborate domain machinery like curated knowledge bases, profiler integrations, and prescriptive workflows.
Now we have deleted the whole framework that may limit models.
A goal-mode agent, a correctness gate, and a four-line instruction telling the model to surpass the best implementation. That's all about Justgoal.
Testbed: FlashAttention, Gated DeltaNet, and DeepSeek Sparse Attention.
2
279
Yuxuan Li retweeted
China’s drone control system has gone insanely next-level. 33,615 drones completed high-precision flight simultaneously, looking like they were building a parallel world in the sky.
This has also refreshed the Guinness World Record that China set before.
Yuxuan Li retweeted
This works really well btw, at the end of your query ask your LLM to "structure your response as HTML", then view the generated file in your browser. I've also had some success asking the LLM to present its output as slideshows, etc.
More generally, imo audio is the human-preferred input to AIs but vision (images/animations/video) is the preferred output from them. Around a ~third of our brains are a massively parallel processor dedicated to vision, it is the 10-lane superhighway of information into brain. As AI improves, I think we'll see a progression that takes advantage:
1) raw text (hard/effortful to read)
2) markdown (bold, italic, headings, tables, a bit easier on the eyes) <-- current default
3) HTML (still procedural with underlying code, but a lot more flexibility on the graphics, layout, even interactivity) <-- early but forming new good default
...4,5,6,...
n) interactive neural videos/simulations
Imo the extrapolation (though the technology doesn't exist just yet) ends in some kind of interactive videos generated directly by a diffusion neural net. Many open questions as to how exact/procedural "Software 1.0" artifacts (e.g. interactive simulations) may be woven together with neural artifacts (diffusion grids), but generally something in the direction of the recently viral teapawt.org/zan2434/status/2046982…
There are also improvements necessary and pending at the input. Audio nor text nor video alone are not enough, e.g. I feel a need to point/gesture to things on the screen, similar to all the things you would do with a person physically next to you and your computer screen.
TLDR The input/output mind meld between humans and AIs is ongoing and there is a lot of work to do and significant progress to be made, way before jumping all the way into neuralink-esque BCIs and all that. For what's worth exploring at the current stage, hot tip try ask for HTML.
Yuxuan Li retweeted
📁 Fei-Fei Li, former Google Chief Scientist, says the industry is dangerously fixated on language models.
Most of the real economy is physical, perceptual and spatial.
Once AI fully understands the visual world, it stops being a chatbot and starts becoming infrastructure.
Yuxuan Li retweeted
Introducing Daybreak: frontier AI for cyber defenders.
Daybreak brings together the most capable OpenAI models, Codex, and our security partners to accelerate cyber defense and continuously secure software.
A step toward a future where security teams can move at the speed defense demands.
