Alex Zhao (@cocohearts) profile picture

waking up research @ oai || mit, imo 2023

Joined January 2024
Photo by Alex Zhao (@cocohearts)Photo by Alex Zhao (@cocohearts)Photo by Alex Zhao (@cocohearts)Photo by Alex Zhao (@cocohearts)
parameter golf was a blast. 2,000+ submissions. 1,000+ verified github accounts. ideas ranging from quantization and depth recurrence to TTT LoRA, SSMs, H-nets, JEPA, and more. autoresearch made iteration dramatically faster — and led to emergent bulletin boards, issue threads, unofficial leaderboards, and agent-built writeups that helped everyone learn from everyone else. it felt like a glimpse of where interaction with AI is headed: humans setting taste and direction, agents helping explore, coordinate, and share what works. our goal was simple: make ml research accessible to anyone, anywhere. it was amazing to see that happen. full recap: openai.com/index/what-parame… future events: jobs.ashbyhq.com/openai/form…
39
50
9
609
170,962
ah yeah
NEWS: Wall Street execs and NSA have told Congress that outages were caused by patching vulnerabilities identified by AI models, sources say AI did not cause outages but found so many issues that the rush to fix them before an attack overwhelmed systems washingtonsun.com/technology…
2
461
in general chinese ai is tough because the chinese enterprise market is small and cost of manufacturing labor is lower at the end of the day chinese models serve american companies and customers, on american chips in america owned by american neoclouds
Just read a Chinese article calling this "the darkest moment for Chinese AI." Seeing pushback in the comments, but basically it lays out the bear case quite extensively: 1. Getting compute is harder whichever way you go. Buying advanced NVIDIA chips is difficult, renting GPUs overseas is coming under pressure, and proposed restrictions on chipmaking equipment could make domestic alternatives harder to produce too. 2. Catching up could get a lot more expensive. If learning from leading models’ outputs aka distillaion becomes harder, Chinese labs have to spend more on their own data, experiments and training. That gets uncomfortable if investors are also losing patience. Independent labs don’t have an advertising or e-commerce business to pay the bills. 3. Robot-brain startups could find their advantage disappearing. Frontier labs are moving into robotics, potentially eating into what specialist models can offer. Meanwhile, robots still have to make economic sense against China’s relatively low labor costs. 4. More users don’t necessarily mean enough revenue. China has huge digital audiences, but getting people and businesses to pay sustainably for AI is another question. Everyone is competing on price, American models are getting cheaper too, and research isn’t getting cheaper. 5. Even the areas where China already leads face risks. Optical-component suppliers are benefiting from AI infrastructure spending, but proposed US restrictions could threaten access to major customers. That’s a lot hitting at once. The worry is that companies keep making better models while needing more money to stay competitive, without earning enough to fund the next round. Now, it feels like every six months it is a "darkest moment" for Chinese AI. But I think these are real concerns. We'll focus on this topic at @techbuzzchina (and present some hope, I think) next week.
9
452
LFG RAJANNNN
i spent the last couple months trying to make big changes to our stack to reduce code slop! while not perfect, i believe 6.2 is a jump in writing more mergable code your feedback means everything to us so keep sharing!
8
790
the conclusion i drew: these are exactly the highest-priority most critical organizations that the USG is privately encouraging maximally rapid patching and recent daybreak/glasswing releases, plus some new tranche of public CVE's, caused lots of patches to all be rushed out at once some of which were migrations that hit minor hiccups, which were all resolved quickly and were NOT cyber incidents
probably worth noticing that venn diagram of firms experiencing major outages and the list of firms with early mythos cyber access is a circle
1
2
23
2,385
Alex Zhao retweeted
there is no bubble, enterprises just adopted ai slower than expected at customers we're seeing creation of terabytes of data every day, thousands of boxes, and 100x pay-willingness for tokens. actually ai-native companies is at ~0% right now and will at some point be 100%
22
21
7
685
37,413
JENSEN HUANG HAS HIRED LAW FIRM PAUL WEISS AFTER HIS EXIT FROM NVIDIA: NYT
7
7
356
14,515
(1/10) Today, we’re sharing new research from @OpenAI: How does CoT monitorability (no-CoT capability and CoT controllability) evolve over the model training lifecycle? We emerged with unsurprising but sobering findings: as we scale toward more intelligent systems, models are becoming harder to monitor through their CoT. 🧵
125
188
75
1,469
272,261
> A decade ago, a big data center in the U.S. was 50 megawatts, and the industry used 5.5 billion gallons of water. The new centers use far less water today to do the same amount of computation, but there are many more of them. finally forbes.com/sites/christopher…
1
21
1,088
Alex Zhao retweeted
ChatGPT and Codex are helping engineers build for the physical world. We’re sharing our progress across mechanical design, electrical engineering, and simulation—from helping our hardware teams iterate faster to testing new workflows with industry collaborators.
735
752
586
8,440
1,423,884
gotta get back on curius
8
598
i've got to try this
openai dot and a 24/7 mac mini have made my life 15% better. couldn’t recommend more. will write a blog post on my setup
2
18
1,637
codex tab autocomplete is brilliant one could just rl the autocompletion model with more ttc to climb acceptance rate combine with a new thread prediction / autocomplete model remote worker ✅
4
33
1,915
Alex Zhao retweeted
The largest math proof in human history is now computer-verified. Today, Axiom announces a complete Lean auto-formalization of the Classification of Finite Simple Groups, the "periodic table of symmetry." About 15,000 pages, built by over 100 mathematicians across generations.
425
1,303
656
7,168
3,750,826
Whatever you believe about the world I think we can all agree that P(P = NP) is increasing....
14
20
5
418
23,024
An internal model at OpenAI has produced a claimed refutation of ETH! Russell Impagliazzo, Mohan Paturi, and I have been asked to help organize an open and independent examination of this claim. 1/4
304
770
544
7,404
3,770,302
i've gone completely numb from the deluge of fucking incredible breakthroughs in the last few months
3
2
55
2,111
aside: i'm really proud of how we're releasing these results in collaboration with the broader scientific community and the scholars that inspired these questions in the first place
4
303
lfg
Replying to @seddonandrew
To show what it can do, here's an example motor controller board, priced on The Grid and at a typical quick-turn shop in Shenzhen.
10
927