
Directing SphereLab @CUHKofficial. Postdoc @MPI_IS. PhD @Cambridge_Uni & @GeorgiaTech. Previous Intern @Google & @nvidia. All opinions are my own.
Joined May 2009
- Tweets1K
- Following799
- Followers2.6K
- Likes3.2K
Weiyang Liu retweeted
🚀 Introducing DeepInfer, our in-house inference framework, now fully open source!
Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.
🔗 github.com/deepseek-ai/DeepI…
1/4
Weiyang Liu retweeted
mHC, iHC, oHC, xHC, and a zoo of AttnRes variants... An LLM architect with FOMO from the daily explosion of new papers? 🫠
Fear not! You can't afford to miss ScalingDepth — our community hub for keeping up with the ever-evolving world of residual connections!
We'll keep tracking and sharing the latest research, methods, and papers from the community 🚀
And it's an OPEN community! Missed something? PRs and DMs are always welcome! Got a cool blog, post, idea, or unexpected finding? We'd love to hear it! ScalingDepth hosts a dedicated space to showcase community insights and discoveries.🌍
Built by the community. Driven by curiosity. Pushing the limits of depth! 🔥
🌐 keyu-wang-2002.github.io/Sca…
Weiyang Liu retweeted
Spent the past three weeks on this idea.
A sampler on a different precision or accelerator is just off-policy, like stale samples in fully async RL, and off-policy correction doesn't care why the sampler differs from the trainer. Meanwhile, inference keeps diversifying across precisions(MXFP8, MXFP4, NVFP4) and accelerators outside RL. So why not use all of that compute for RL rollout?
The sampler and the trainer are two systems built for different purposes. Hope this opens up new possibilities for RL algorithms and systems, and leads to cleaner system designs. Enjoy the blog, and stay tuned for the next one on our infra :)
Training large models with RL typically runs on a fixed homogeneous cluster of the latest accelerators for numerical stability and faster throughput. This throws away inference optimization work done for other vendors and generations, limits the capacity you can utilize, and is unnecessarily expensive.
So we trained a model on agentic coding tasks using fully asynchronous RL and mixed-precision, mixed-accelerator samplers. With off-policy correction, training stayed stable through convergence, on par with homogeneous samplers, at a lower cost.
Weiyang Liu retweeted
I was among the first people to work on Google DeepMind's AI safety team, as an intern in 2018.
My coworkers wanted me to quit my PhD and continue working there full time.
I chose not to.
My reasons included:
1) I was told not to mention YouTube in my research about manipulative AI algorithms. I didn't want to have to censor myself.
2) I did not believe Google, or any of the other AI companies, really understood and cared about safety.
3) I have never been optimistic about the prospects for a technical solution to the problems posed by AI.
We need a governance solution.
At this point, we've waited too long for something nuanced as the first line of response. It's an urgent crisis, and we need to stop AI now.
So many of my peers have joined AI companies, knowing the risks involved if this technology continues to advance.
It's great to see when someone at one of these companies realizes we're not on track to survive this, and that they don't want to be complicit in murdering billions, and quits.
But please remember that these are people who made a bad bet in choosing to work at these companies.
They thought the companies would be responsible.
They were wrong.
They thought we'd solve alignment in time.
That also looks unlikely.
Many of them thought that we'd never be able to stop AI development.
That remains to be seen, but it's looking a lot more likely than anyone thought.
I do sometimes wish that I'd chosen to work at a frontier AI company just so I could quit dramatically to say what really needs to be said:
Not just that this technology is dangerous.
Not just that the risk of human extinction is real.
Not just that the companies are irresponsible.
But also:
We need to stop.
There is no alternative.
I am not a doomer. I think we could get lucky and survive. But I'd be surprised.
I am not exaggerating for effect.
I am not saying this to be a downer or a hater.
I am not being idealistic -- I'm being realistic and pragmatic.
And I am not talking out of my ass. I've been in this field since before these companies existed.
To everyone who is new to this, or who is jumping on this story and wondering what to make of it: This is your answer. We MUST stop. Now.
Weiyang Liu retweeted
We're publishing our most detailed threat intelligence report to date.
It covers how people tried to misuse Claude—for cyberattacks, influence operations, surveillance, biology, and building weapons—and how we found and stopped them.
We disrupted every operation in the report, and used the lessons from them to strengthen our safeguards. Where appropriate, we also shared what we found with authorities and other AI companies.
These cases are not typical: we’re highlighting some of the most sophisticated misuse we’ve seen. But they’re especially important to discuss, because they show us where AI misuse is headed, where our safeguards work, and where they need to improve.
We’re publishing this report so others can spot the same activity on their own platforms, and so we can give the public a clearer view of how emerging threats develop.
Read the report: anthropic.com/threat-intelli…
Weiyang Liu retweeted
We can finally talk about it:
We found a way to extract hidden reasoning of frontier models using a vulnerability in the APIs of every frontier AI company.
We verified that our reasoning token count matches billed API thinking tokens 1:1 for most of the prompts we queried.
Weiyang Liu retweeted
arxiv.org/abs/2608.01284
nGPT training recipe. Turns out that it requires many tricks (and some people keep trying it).
Weiyang Liu retweeted
Songlin Yang's video explanations
youtube.com/watch?v=d0HJvGSW…
youtube.com/watch?v=RTJKXK5L…
Songlin Yang's blog post
Design intuition
sustcsonglin.github.io/blog/…
Kernel algorithm (I skipped to "A Chunkwise Algorithm for DeltaNet" section)
sustcsonglin.github.io/blog/…
FlashKDA
github.com/MoonshotAI/FlashK…
I prompted Codex to explain it to me, only after that the deep dive made sense
github.com/MoonshotAI/FlashK…
vLLM serving explanations
vllm.ai/blog/2026-07-22-kimi…
vllm.ai/blog/2026-07-27-k3#p…
The great Zhihu post for explaining AttnRes
Training
zhihu.com/question/201699309…
Inference
zhuanlan.zhihu.com/p/2017528…
Jianlin Su's MoE 環遊記 series
kexue.fm/tag/moe/1/
Explains the concept of MoE from math first principles
Papers
Kimi Linear, LatentMoE, Kimi K3 tech report
N/
Weiyang Liu retweeted
🤯 This is a website, a simple web-based game built with WebGL and Three.js.
Website: messenger.abeto.co
It's honestly surprising how far web development has come.
Weiyang Liu retweeted
The bitter lesson in 26 words:
Don’t be distracted by human knowledge, as AI has been historically.
Instead focus on methods for creating knowledge that scale with computation, like search and learning.
Weiyang Liu retweeted
🚀 DeepSeek-V4 Preview is officially live & open-sourced! Welcome to the era of cost-effective 1M context length.
🔹 DeepSeek-V4-Pro: 1.6T total / 49B active params. Performance rivaling the world's top closed-source models.
🔹 DeepSeek-V4-Flash: 284B total / 13B active params. Your fast, efficient, and economical choice.
Try it now at chat.deepseek.com via Expert Mode / Instant Mode. API is updated & available today!
📄 Tech Report: huggingface.co/deepseek-ai/D…
🤗 Open Weights: huggingface.co/collections/d…
1/n
Weiyang Liu retweeted
10 years ago today, we lost Sir David MacKay FRS. Physicist. Mathematician. Polymath. Gone at 48. I was working on my PhD at Cambridge, and attended some of his last lectures and symposium. He was a reason that attracted me to Cambridge over MIT in 2014.
His textbook, Information Theory, Inference, and Learning Algorithms, was the first ML book I ever read — recommended to me by none other than Geoff Hinton.
He used that same information theory to build Dasher — a text entry system where users steer through a continuous stream of letters flowing toward them, with a probabilistic language model making likely next letters larger and easier to reach, so that any tiny movement — a finger, a gaze — becomes efficient writing. It was the first ML application that truly blew my mind, and sent me deep into a rabbit hole: arithmetic coding, PAQ8 compression, nonparametric models. A journey I partly owe to his PhD student Christian Steinruecken, who also happened to share my love of Japan.
As Chief Scientific Advisor to the UK's Department of Energy & Climate Change, he brought a physicist's clarity to policy. In Sustainable Energy – Without the Hot Air, he ran the numbers on our entire energy diet — and made me confront an uncomfortable truth. One of the biggest single factors? Beef — roughly 1,000 days of cow-time per steak. Hard to argue with the data. Hard to act on it when you were born and raised in Japan. I'm still working on that one, David.
At his final symposium in Cambridge — just a few weeks before his passing — the room told the full story. Geoff Hinton and his Caltech PhD advisor John Hopfield — both Nobel Prize winners in Physics 2024 — gave tributes. Environment policy advisers spoke. Dasher users sent video messages of thanks from around the world — people who found their voice because of him. It was extraordinary to witness, in one room, just how many minds and lives a single person had touched.
The story of how Hinton first noticed him: at a conference workshop poster session, among everyone who stopped by, it was the young MacKay who asked the sharpest, most penetrating question. Hinton remembered it. That's how it begins.
I've always liked physicists who cross into ML — they bring a groundedness, a refusal to hide behind formalism without meaning. David MacKay and Max Welling are the role models I point to. Not just for the mathematics they built, but for how they carried it: with humility, curiosity, and a stubborn insistence on reaching beyond academia.
He seemed to know his time was limited, and gave everything anyway. His legacy stays.
Weiyang Liu retweeted
“There is geometry in the humming of the strings. There is music in the spacing of the spheres.”
— Pythagoras
Weiyang Liu retweeted
Coming soon: #ICCV2027 Hong Kong 🇨🇳
Weiyang Liu retweeted
With Emmanuel Dupoux scp.net/persons/dupoux/ and Yann LeCun @ylecun, we consider a cognitive science inspired AI. We analyse how autonomous learning works in living organisms, and propose a roadmap for reproducing it in artificial systems.
lnkd.in/eNWDmuqT



































