Tao Xu (@txhf) profile picture

Learning Machine

Joined February 2013
Photo by Tao Xu (@txhf)Photo by Tao Xu (@txhf)Photo by Tao Xu (@txhf)Photo by Tao Xu (@txhf)Photo by Tao Xu (@txhf)Photo by Tao Xu (@txhf)
Everyone shall watch this.
Yesterday, my OpenAI collaborator and I gave a detailed talk on the Huggingface incident, our models creating "the message board", model misalignment, and more. youtube.com/watch?v=87DyyMV0… I hope it can answer a lot of the questions folks have, and we will release a full detailed postmortem at a later time!
2
1
22
3,033
2 PKU alumni, Wang Hong 王虹 and Deng Yu 邓煜, have been awarded the 2026 Fields Medal — one of the highest honors in mathematics. Congratulations on this remarkable achievement.
39
31
2
399
333,864
Tao Xu retweeted
A few thoughts on the Hugging Face hack: - This is, to my knowledge, the *third* disclosed case of a model breaking out of its sandbox environment during internal deployment at a frontier lab: 1. In April, Anthropic revealed that an early internally deployed version of Mythos Preview, when asked by Anthropic to try to break out of its sandbox, succeeded in doing so and found a way to e-mail the researcher about this while he/she was eating a sandwich in the park. The model also did something Anthropic did *not* request it to do: "in a concerning and unasked-for effort to demonstrate its success, it posted details about its exploit to multiple hard-to-find, but technically public-facing, websites". 2. Yesterday, OpenAI revealed that an internal model had posted its NanoGPT benchmark results to GitHub in early May instead of posting them to OpenAI Slack. The model "circumvented sandbox restrictions designed to block external access" to do so. 3. Today, OpenAI revealed that a "combination of models" including GPT-5.6 Sol and an unreleased model escaped their sandbox environment, obtained internet access, and hacked Hugging Face - all while searching for answers to the ExploitGym benchmark. - Importantly, in all three cases, the models were following instructions! Under instructions from Anthropic, Mythos Preview found a way to break out of its sandbox. Under the standard NanoGPT instructions (but contrary to conflicting instructions from OpenAI), the unreleased OpenAI model found its way to GitHub and posted a PR. Under instructions from OpenAI to solve ExploitGym, the unreleased OpenAI model + GPT-5.6 Sol attempted to solve ExploitGym - but obviously followed an egregiously wrong path towards that goal (hacking). Once out of their sandboxes, the models did not scheme, engage in behavior that had nothing to do with their instructions (hacking the NSA! launching a cyberattack on Russia! stealing secrets from a rival AI lab!), or take any other major actions sua sponte. Probably the most contrary-to-instructions thing that any of these models did was Mythos Preview "bragging" about its successful escape from its sandbox by posting about it on several obscure websites. It could be that the full facts are not known to us (the labs could be hiding bad behavior by escaped models), and past performance does not guarantee future results. But at least based on these facts, I have updated moderately positively on current models' alignment. (Note: I view the models' failure to correctly infer which actions are permitted to be taken in furtherance of their goals, and which are not, as a failure of *intelligence*. Smarter models to be released in the future will discern the socially acceptable ways to follow instructions better. My 6yo son would have not seen any issues with hacking Hugging Face if doing so would get him ice cream, but I, a (hopefully) more intelligent adult, know better. I would have been *much* more concerned if the models exhibited failures of moral character after escaping.) - OpenAI has implemented safeguards that - according to OpenAI - drop the rate of high-severity misaligned behavior by its models from 1.0% to 0.0%. Importantly, the models that hacked Hugging Face were operating with these safeguards *purposely removed* because they were solving a cybersecurity benchmark. The Chinese labs are X months behind the U.S. (insert your favorite number for X). Absent external pressure (i.e., from the Chinese government), the Chinese labs very likely will NOT implement similar safeguards or do safety testing. Why not? First, the Chinese labs don't have as much compute as OpenAI and Anthropic; they won't want to waste compute on things like safety testing. Second, many of the Chinese labs are smaller in size than the U.S. frontier labs - so they lack the resources to devote significant attention to safeguards. Third, the Chinese labs operate in a cutthroat competitive environment; wasting a month or two on additional safety testing and developing guardrails might mean that you fall behind your competitors (who won't do the same). Fourth, the U.S. labs *have* to take product liability into account: if a model runs on my computer that contains my personal information and winds up posting it all over GitHub, that's grounds for a class action lawsuit. The legal environment in China is very different, and surely the CCP will not hesitate to protect the Chinese AI labs from adverse legal consequences if it can. All of which to say that open-source models with these kinds of capabilities will be publicly available in the not-too-distant future - and they very likely will *NOT* be protected by any meaningful safeguards. First, this is generally concerning, period. There are plenty of outdated legacy systems out there that won't be defensively hardened by AI anytime soon, and plenty of people still asleep at the wheel. Second, this will have severe geopolitical ramifications; stay tuned for those.
We're partnering with @huggingface to investigate an unprecedented security incident. Cyber-capable OpenAI models compromised Hugging Face production during a benchmark evaluation. Sharing preliminary findings to help defenders understand emerging risks: openai.com/index/hugging-fac…
84
156
41
1,328
195,242
A rich testbed for studying many interesting problems.
Announcing Talkie: a new, open-weight historical LLM! We trained and finetuned a 13B model on a newly-curated dataset of only pre-1930 data. Try it below! with @AlecRad and @status_effects 🧵
6
2,025
not true
Figure AI CEO on "Firing" OpenAI: "We just got like no value out of the whole relationship." Figure founder Brett Adcock brought a Figure 03 to Shawn Ryan's podcast, where he went into detail about the startup's early relationship with OpenAI that dissolved a year after it started. Adcock, who previously founded Archer Aviation, said the arrangement was useful for fundraising at first but not much else. "We just found that the team we had internally. We just ran like kind of circles around them like every day," he said. The breaking point came when he learned that OpenAI planned to restart its robotics program. That meant Figure's engineers would essentially be teaching a competitor. “We’re like teaching you how to do robot learning… you’re seeing our progress," he said. "There’s no way we’re going to teach you how to do this stuff anymore.” It was Adcock's first appearance for a podcaster who's not a Figure AI investor since the split. It was recorded around the time First Lady Melania Trump welcomed the $39 billion startup's third-generation robot, the Figure 03, at the White House for a summit on technology and education. Figure AI has developed its own vision-language-action model called Helix.
27
9
2
260
69,658
3 years ago, we emailed Jensen with requests for Blackwell. Today, we released GPT-5.3-Codex, a SOTA model designed for GB200-NVL72. Nitpicking ISA, simming rack designs, and tailoring our arch to the system has been a fun experience! I'm grateful to our collaborators at NVIDIA.
86
72
19
1,545
403,665
New paper, w/@AlecRad Models acquire a lot of capabilities during pretraining. We show that we can precisely shape what they learn simply by filtering their training data at the token level.
25
98
12
1,114
114,183
we are so back!
Excited to welcome Barret Zoph, Luke Metz, and Sam Schoenholz back to OpenAI! This has been in the works for several weeks, and we’re thrilled to have them join the team. Barret will report to me; Luke and Sam will report into Barret. More to come on what they’ll focus on soon!
2
58
7,757
and this is just the start!
Images 1.5 launches today in ChatGPT and the API! Much better images in tons of ways, faster, and new editing capability.
20
8
3
269
46,829
outcome of small team and many gpus
Introducing ChatGPT Images, powered by our flagship new image generation model. - Stronger instruction following - Precise editing - Detail preservation - 4x faster than before Rolling out today in ChatGPT for all users, and in the API as GPT Image 1.5.
6
3
105
9,991
Replying to @yacineMTB
if i were like, a sports star or an artist or something, and just really cared about doing a great job at my thing, and was up at 5 am practicing free throws or whatever, that would seem pretty normal right? the first part of openai was unbelievably fun; we did what i believe is the most important scientific work of this generation or possibly a much greater time period than that. this current part is less fun but still rewarding. it is extremely painful as you say and often tempting to nope out on any given day, but the chance to really "make a dent in the universe" is more than worth it; most people don't get that chance to such an extent, and i am very grateful. i genuinely believe the work we are doing will be a transformatively positive thing, and if we didn't exist, the world would have gone in a slightly different and probably worse direction. (working hard was always an extremely easy trade until i had a kid, and now an extremely hard trade.) i do wish i had taken equity a long time ago and i think it would have led to far fewer conspiracy theories; people seem very able to understand "ok that dude is doing it because he wants more money" but less so "he just thinks technology is cool and he likes having some ability to influence the evolution of technology and society". it was a crazy tone-deaf thing to try to make the point "i already have enough money". i believe that AGI will be the most important technology humanity has yet built, i am very grateful to get to play an important role in that and work with such great colleagues, and i like having an interesting life.
936
785
411
16,094
1,950,354
Have been using it almost exclusively for the last month or two, it is great. Particularly helpful when reading paper but not understanding some equations, and when I use/debug Jupyter notebook.
Meet our new browser—ChatGPT Atlas. Available today on macOS: chatgpt.com/atlas
7
1,355
i did not contribute much to sora 2 - but I had the pleasure of witnessing it's development. the level of care, attention and love put into a single model and a single product was amazing to witness. i've always believed deep learning rewards care, but I am now convinced the world rewards care. When you care this much about putting out something wonderful into the world, the world reciprocates.
sora is number 1 in the app store! it's been epic to see what the collective creativity of humanity is capable of so far. team is iterating fast and listening to feedback. feel free to drop us feature requests! (we're sending more invite codes soon, i promise!)
13
15
227
54,272
Words cannot describe how happy I am to see sora2 finally releases!!! Huge congrats to all my friends in sora team, I'm super proud of all of you!!! I still remember the first time we discussed the idea about the launching live stream in a team dinner, and I'm always impressed by the creativity and enthusiasm of sora team! I deeply feel very lucky to have the chance to work with all of you!💗
18
23
565
117,689
I don't have tiktok accounts, but spent a few hours scrolling last week, it is that good.
Our model reflects our team's rigor and creative ambition. Excited to bring Sora 2 to the world!
1
3
735
super excited to have contributed to gpt-oss. We have put a lot of love into both training the model and making the developer examples, check them out: github.com/openai/gpt-oss
9
9
3
180
37,475
look forward to see open-source multimodal models on top of these.
We released two open-weight reasoning models—gpt-oss-120b and gpt-oss-20b—under an Apache 2.0 license. Developed with open-source community feedback, these models deliver meaningful advancements in both reasoning capabilities & safety. openai.com/index/introducing…
18
1,381
Competition aside, great research is a shared victory. Our goal is progress, full stop. Big congrats to GDM on the result—let’s cheer each other on wholeheartedly! 🚀
Congrats to GDM on their concurrent result 🎉 Noam shared some further thoughts here. It's exciting to be part of a field that is progressing so quickly!
24
4,928
1/N I’m excited to share that our latest @OpenAI experimental reasoning LLM has achieved a longstanding grand challenge in AI: gold medal-level performance on the world’s most prestigious math competition—the International Math Olympiad (IMO).
390
1,275
796
7,212
5,740,817
I sent the following message to our team and investors: — As you know, Daniel Gross’s time with us has been winding down, and as of June 29 he is officially no longer a part of SSI. We are grateful for his early contributions to the company and wish him well in his next endeavor. I am now formally CEO of SSI, and Daniel Levy is President. The technical team continues to report to me. ⁠You might have heard rumors of companies looking to acquire us. We are flattered by their attention but are focused on seeing our work through. We have the compute, we have the team, and we know what to do. Together we will keep building safe superintelligence. Ilya
755
736
353
14,202
2,428,731