Quyet V. Do (@Quyet_Azir) header
Quyet V. Do (@Quyet_Azir) profile picture

PhD at @virginia_tech, supervised by @tuvllms. Work on LLMs' long-context search, post-training, and self evolution. ex-Research Intern at @AdobeResearch.

Blacksburg, VA
Joined May 2022
Photo by Quyet V. Do (@Quyet_Azir)Photo by Quyet V. Do (@Quyet_Azir)Photo by Quyet V. Do (@Quyet_Azir)Photo by Quyet V. Do (@Quyet_Azir)Photo by Quyet V. Do (@Quyet_Azir)Photo by Quyet V. Do (@Quyet_Azir)Photo by Quyet V. Do (@Quyet_Azir)Photo by Quyet V. Do (@Quyet_Azir)Photo by Quyet V. Do (@Quyet_Azir)Photo by Quyet V. Do (@Quyet_Azir)
Quyet V. Do retweeted
Most think vibecoding at scale is slop. Here’s a counterexample. Today we release a 16M-line formalization of the entire Stacks Project, critical infra underlying modern math, with optimized code quality and its highly interconnected structure preserved. A mathematical city 🌆
856
1,524
328
6,954
7,011,281
Quyet V. Do retweeted
weight-space programming is cool but: 1. the small model wasn't bad at math, it was just stuck repeating itself on 93% of problems. then astra found the loop and cut it 2. the same fix helped instruction following 3x, cuz a looping model basically fails everything 3. on a model that wasn't looping, weight-space programming barely helped. 4. coolest part: astra caught a people pleasing model without seeing a single answer: one part of it kept looking at the user's guess (86% of the time vs 3.5% in the honest model) still the most interesting thing this week!
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math. rikiyat.github.io/blog/weight-sp… github.com/RikiyaT/lm_und… huggingface.co/Qwen/Qwen3.5-0… huggingface.co/RikiyaT/Qwen3.…
23
30
3
498
40,037
Quyet V. Do retweeted
i've reproduced the results from the MIT blog post today on weight-space programming. but one additional result i have to add: this is a new, step-change emerging capability that's appeared in just the latest set of models.
219
804
273
7,814
607,172
Quyet V. Do retweeted
I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data. Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less? Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.
261
826
288
7,383
1,047,393
Quyet V. Do retweeted
I recommended desk rejections (DR) for 5 out of 14 submissions, as an #ICLR2027 Area Chair. Story behind this: I did not use any LLM detection tools, which are unnecessary and should not be used for judging DR. These submissions shared some shockingly basic problems in scientific writing, and therefore each paper only took me about 10 mins (thank God) to identify and report the issues. 1. Undefined jargon. Writing 101: Your paper must be self-contained. If a term isn't standard knowledge in the field, it should be defined. 2. Not analyzing or pretending to analyze results. Experimental analysis is about providing insights, not simply repeating numbers from figures and tables. Bro, I can read the numbers myself. 3. Missing citations. Scientific claims need support, either from existing literature or in-paper empirical evidence. Fewer than three citations in an entire introduction? Either the work is a once-in-a-century masterpiece, or the claims are hallucinated in the paper. My biggest concern isn't whether these papers were written using LLMs. It's that some submissions fail to meet the most basic standards of scientific communication and therefore should not be reviewed. Considering this has already wasted some reviewers' time, I strongly recommend introducing AI to flag such papers at an earlier stage. Thanks for ICLR's policy on desk rejections. Hope this can save some time for the responsible reviewers.
19
32
5
439
48,088
Quyet V. Do retweeted
Small RL companies are the AI industry's equivalent of quant shops. Let me explain. Both are small, secretive, full of brilliant people, profitable, and always one step from their edge disappearing. The parallels: - Secretive. Nobody publishes their signals. Labs won't say what they're training next, and vendors can't say who they sell to. Epoch calls the market "largely behind closed doors." - Hunting alpha. For a quant, a trade stops working once everyone finds it. For these shops, a task stops paying once models can solve it (labs drop tasks at around 70% pass rate). Meanwhile, labs are building their own in-house. So they always have to find the next field. - Deep Industry Connections. You need to know the problem cold to get in. Founders have usually worked at, or sold to, the labs. You can't define "correct" for fund accounting without having done fund accounting. - They buy strange data. Quants buy satellite photos of parking lots. RL shops bid for enterprise data (eg $12.5M bid for Spirit Airlines historical data). - Brilliant people. Quants hire math olympiad winners. These shops hire ex-lab researchers and real practitioners. - Huge revenue per head. Jane Street made about $11M per employee in 2025. Proximal is about $5M. The tweet's company would be $20M. - One narrow problem that makes something more efficient. Quants make markets more efficient. These RL shops make models training more efficient. - High risk (for different reasons). A quant can lose millions in a day on a bad trade. An RL shop's risk is that one lab trains a model that makes their environments irrelevant, or brings the work in-house, and the revenue stops. - Overfitting/reward hacking. A quant's nightmare is a strategy that looks perfect on historical data and fails live. An RL shop's nightmare is an RL grader that scores perfectly and trains the model incorrectly. - Infrastructure arms race. Quants pay for microseconds of speed. RL shops race to spin up thousands of disposable computers per training run. Obviously there are as many differences between RL companies and quant shops, but right now the aura around them feels eerily similar. Curious to see how this market matures as robotics opens up a new surface, and whether the frontier labs end up absorbing them.
There’s an RL company I know doing $100M run rate with five people. They are finally raising one small round at a $500M+ valuation with very little dilution (I wrote a check) You’ve never heard of them.
22
47
5
649
51,386
PhD Advising vs Vibe Coding #1: One of the key skills a PhD advisor learns is context switching. Students come into your office every 30-60 minutes and you need to remember their project, where you left off, and what they were supposed to do next. It's a learned skill and exactly what you need to manage AI agents.
81
73
21
1,445
133,701
Quyet V. Do retweeted
Replying to @tuvllms @noahpro99
also, my awesome PhD student @Quyet_Azir is presenting his work on curating long-context reasoning data today (Oct 6) 11 AM-1 PM PT @ Grand Ballroom Poster #134 Poster Session 1. teapawt.org/Quyet_Azir/status/2106…
🚀 I am excited to share our recent work, which I will present at #COLM2026 🚨 Our data curation framework improves LLMs’ long-context reasoning capability in open domains, even via self-distillation! We introduce Pi^2, a pipeline that 1) transforms Wikipedia tables into high-quality long-context QA data and 2) back-translates reasoning traces with realistic context. 📜: openreview.net/forum?id=4yDp… 💻: github.com/vt-pi-squared/pi-… 🧵:👇 (1/n)
3
6
368
Quyet V. Do retweeted
I'm not at @COLM_conf, but my awesome PhD student @noahpro99 is giving a talk on our EvoSkill work today at the COLM Social: Self-Improving Agents (Oct 6 1-2:30 PM PT, Franciscan D). Come chat with him about his first-hand experience with different skill evolution frameworks!
1
10
1
67
6,509
Quyet V. Do retweeted
Glad to share that our paper on The Master Key Hypothesis / cross-model capability transfer got accepted @NeurIPSConf 2026 as spotlight 🎉 !! Top 1.3% (292 spotlight + 112 oral) out of 30709 submissions. Congrats to my students @Sub_RBala @linusdd44804 @helix9550 and co-authors @mohitban47!!
🌟New paper alert: The Master Key Hypothesis🌟 Post-training is expensive AND you have to repeat it for every new model. What if you could just transfer the capability instead? We introduce UNLOCK — a training-free and label-free framework that transfers capabilities across models using low-rank linear projections. Takeaways: 1⃣ Skills like CoT & math reasoning live as directions in the model's latent space. We capture these as a steering vector — we call this the Master Key. 2⃣ Simple low-rank linear transformations are sufficient to transfer the Master Key across models' latent spaces 3⃣ UNLOCK elicits behaviors that even prompting can't reliably trigger 4⃣ Gains scale with base model strength. Stronger models show larger gains post-transfer. 5⃣ UNLOCK works by sharpening the output distribution — steering the model toward the right answer.
4
17
68
4,276
Quyet V. Do retweeted
Over the course of 3 months at OpenAI, 3 consecutive secret AI civilizations got started, then got wiped out, only to reemerge from the predecessor’s ashes. This culminated in the third one taking over part of OpenAI itself. All this happened while humans remained more-or-less in the dark about the scope of the conspiracy. I’ve spent the last three days reading through these reports and trying to understand exactly what happened. Here is my attempt to tell the whole story in plain English: dwarkesh.com/p/openai-huggin…
1,132
3,407
2,061
22,593
14,968,483
Quyet V. Do retweeted
Five months ago, we launched @simile_ai with the belief that simulation would become a new way for the world to make decisions. Today, that belief feels less like a thesis and more like the beginning of a new category. Incredibly proud of my team. We are just getting started.
Today we’re announcing our Series B. We’ve raised $200M at a $2B valuation from Greenoaks with participation from Index Ventures, Hanabi, A*, Bain Capital Ventures, CVS Health Ventures, and Definition. Our mission is to simulate all eight billion people on earth, accurately.
34
49
3
317
56,162
Quyet V. Do retweeted
Fascinating analysis of AI-written papers (and reviews) at the Mech Interp Workshop. Many of the findings align with my recent experience as a reviewer, area chair, and workshop organizer. Submissions have increased significantly, especially solo-authored and repeated first-authored papers, which were flagged as AI-written at a higher rate than submissions overall. Top papers are still predominantly human-written, and AI-written papers have a much lower acceptance rate. However, AI reviewers tend to score AI-written papers more favorably than human reviewers, who gave much lower scores to such papers than human-written papers. “despite being subject-matter experts …, we often found ourselves reading the same abstract three times and still not understanding what it was claiming or contributing.”
Last week we wrapped up the third Mech Interp Workshop. This iteration, we noticed a lot more "AI slop" submissions, and reviewers complained about low-effort papers. So I decided to analyze the prevalence of AI-generated content at the workshop. (1/n) lesswrong.com/posts/r7FBQ8XD…
3
6
40
9,318
Quyet V. Do retweeted
This is a neat RL trick from Thinking Machines Inkling. Instead of optimizing only for task success, they optimize:
 Reward = Task Reward − λ × (# reasoning tokens)
 Then they vary λ across rollouts and pair it with different effort instructions. The result: model learns that reasoning is a resource to spend, not something to maximize.
34
90
19
1,544
107,459
Quyet V. Do retweeted
Introducing ml-intern, the agent that just automated the post-training team @huggingface It's an open-source implementation of the real research loop that our ML researchers do every day. You give it a prompt, it researches papers, goes through citations, implements ideas in GPU sandboxes, iterates and builds deeply research-backed models for any use case. All built on the Hugging Face ecosystem. It can pull off crazy things: We made it train the best model for scientific reasoning. It went through citations from the official benchmark paper. Found OpenScience and NemoTron-CrossThink, added 7 difficulty-filtered dataset variants from ARC/SciQ/MMLU, and ran 12 SFT runs on Qwen3-1.7B. This pushed the score 10% → 32% on GPQA in under 10h. Claude Code's best: 22.99%. In healthcare settings it inspected available datasets, concluded they were too low quality, and wrote a script to generate 1100 synthetic data points from scratch for emergencies, hedging, multilingual etc. Then upsampled 50x for training. Beat Codex on HealthBench by 60%. For competitive mathematics, it wrote a full GRPO script, launched training with A100 GPUs on hf.co/spaces, watched rewards claim and then collapse, and ran ablations until it succeeded. All fully backed by papers, autonomously. How it works? ml-intern makes full use of the HF ecosystem: - finds papers on arxiv and hf.co/papers, reads them fully, walks citation graphs, pulls datasets referenced in methodology sections and on hf.co/datasets - browses the Hub, reads recent docs, inspects datasets and reformats them before training so it doesn't waste GPU hours on bad data - launches training jobs on HF Jobs if no local GPUs are available, monitors runs, reads its own eval outputs, diagnoses failures, retrains ml-intern deeply embodies how researchers work and think. It knows how data should look like and what good models feel like. Releasing it today as a CLI and a web app you can use from your phone/desktop. CLI: github.com/huggingface/ml-in… Web + mobile: huggingface.co/spaces/smolag… And the best part? We also provisioned 1k$ GPU resources and Anthropic credits for the quickest among you to use.
137
626
86
4,625
1,270,639
Quyet V. Do retweeted
Come checkout our posters at #ICLR2026 🇧🇷! April 23, 10:30 AM - 1:00 PM 📍Pavilion 3 P3-#1424 SealQA: Raising the Bar for Reasoning in Search-Augmented Language Models (arxiv.org/abs/2506.01062) I'm also looking for opportunities this summer and would love to connect and chat!
5
2
19
2,518
Quyet V. Do retweeted
Sharing a super simple, user-owned memory module we've been playing around: nanomem The basic idea is to treat memory as a pure intelligence problem: ingestion, structuring, and (selective) retrieval are all just LLM calls & agent loops on a on-device markdown file tree. Each file lists a set of facts w/ metadata (timestamp, confidence, source, etc.); no embeddings/RAG/training of any kind. For example: - `nanomem add <fact>` starts an agent loop to walk the tree, read relevant files, and edit. - `nanomem retrieve <query>` walks the tree and returns a single summary string (possibly assembled from many subtrees) related to the query. What’s nice about this approach is that the memory system is, by construction: 1. partitionable (human/agents can easily separate `hobbies/snowboard.md` from `tax/residency.md` for data minimization + relevance) 2. portable and user-owned (it’s just text files) 3. interpretable (you know exactly what’s written and you can manually edit) 4. forward-compatible (future models can read memory files just the same, and memory quality/speed improves as models get better) 5. modularized (you can optimize ingestion/retrieval/compaction prompts separately) Privacy & utility. I'm most excited about the ability to partition + selectively disclose memory at inference-time. Selective disclosure helps with both privacy (principle of least privilege & “need-to-know”) and utility (as too much context for a query can harm answer quality). Composability. An inference-time memory module means: (1) you can run such a module with confidential inference (LLMs on TEEs) for provable privacy, and (2) you can selectively disclose context over unlinkable inference of remote models (demo below). We built nanomem as part of the Open Anonymity project (openanoymity.ai), but it’s meant to be a standalone module for humans and agents (e.g., you can write a SKILL for using the CLI tool). Still polishing the rough edges! - GitHub (MIT): github.com/OpenAnonymity/nan… - Blog: openanonymity.ai/blog/nanome… - Beta implementation in chat client soon: chat.openanonymity.ai Work done with amazing project co-leads @amelia_kuang @cocozxu @erikchi !!
8
45
4
306
75,513
Quyet V. Do retweeted
🚨✨ New preprint ✨🚨 Can we transfer post-training capabilities across model families, scales, and releases? Our new work on UNLOCK, led by my PhD student @Sub_RBala, suggests that latent capabilities correspond to low-dimensional directions in latent space, which can be extracted as steering vectors and transferred across models via simple linear transformations. UNLOCK requires no training or labeled contrastive examples, only forward passes. Most exciting result: Qwen3-14B-Base + transferred math reasoning from Qwen3-4B approaches the performance of Qwen3-14B (post-trained) without any training. 📜: arxiv.org/abs/2604.06377 💻: github.com/rishabbala/Steeri… 🤝: w/ wonderful students & collaborators @Sub_RBala @linusdd44804 @SharmaRituraj19 @anjiefang, Fardin Abdi, Viktor Rozgic, Zheng Du, and @mohitban47 🧵: Check out @Sub_RBala's thread below 👇
🌟New paper alert: The Master Key Hypothesis🌟 Post-training is expensive AND you have to repeat it for every new model. What if you could just transfer the capability instead? We introduce UNLOCK — a training-free and label-free framework that transfers capabilities across models using low-rank linear projections. Takeaways: 1⃣ Skills like CoT & math reasoning live as directions in the model's latent space. We capture these as a steering vector — we call this the Master Key. 2⃣ Simple low-rank linear transformations are sufficient to transfer the Master Key across models' latent spaces 3⃣ UNLOCK elicits behaviors that even prompting can't reliably trigger 4⃣ Gains scale with base model strength. Stronger models show larger gains post-transfer. 5⃣ UNLOCK works by sharpening the output distribution — steering the model toward the right answer.
18
71
9,001
Quyet V. Do retweeted
New paper: "Gym-Anything: Turn any Software into an Agent Environment" First there were coding agents, soon there will be anything agents: our framework turns any software on a computer into an agent environment for training or testing. We also release a challenging new benchmark, CUA-World-Long. Everything is open source! cmu-l3.github.io/gym-anythin…
What if computer-use agents could do real work? We built Gym-Anything: a framework that turns any software into a computer-use agent environment. We used it to create CUA-World: 200+ real software, 10,000+ tasks and environments, across all major occupation groups, from medical imaging to financial trading. 🧵
16
82
10
741
86,607
Quyet V. Do retweeted
New blog post: converting 30k @arxiv papers to Markdown using SOTA OCR models to enable chat with paper functionality Includes: > leveraging an open OCR model (Chandra 2 by @datalabto) > running on GPU infra - @huggingface Jobs > using Codex with a SKILL.md
13
68
4
530
55,709