Head of Realtime AI @OpenAI. Created WebRTC. Past: Distinguished Engineer @google, AIM. Amateur mathematician/musician.

Seattle, WA
Joined February 2007
Just posted our technical deep dive on the GPT-Live system, by @zahanm and yours truly.
GPT-Live can listen while it speaks. To make that feel natural at ChatGPT scale, we rebuilt the voice stack from client to model. This new architecture keeps audio flowing continuously, so deeper reasoning and tool use don't interrupt the conversation.
34
37
9
663
167,915
It's good to be cautious here, but these arguments are surprisingly hand-wavy: 1) intelligence doesn't imply consciousness 2) we think computational functionalism is "fatally flawed" 3) some theories require specific physical properties that LLMs lack
3
1
17
2,061
Justin Uberti retweeted
Dots, now on your Home Screen
113
33
16
986
362,951
Nonlinear activation functions can do some amazing things!
My dear friends, I’d like to show you my latest toy project Introducing Handwired: turn your regular JavaScript code into a neural net! - no training - exact outputs vs the JS program - fully differentiable All runnable in a browser, without GPU
1
10
1,757
Absolutely insane live demo. So many things that could have gone wrong. But they nailed it.
Real life, baby
2
1
38
2,586
If your Live API app typically selects from a few specific tools, you can now use the new Decisions API to speed up delegation, while retaining the ability to use a full LLM for long-tail queries.
you can now use native decision delegation in your voice app. a week ago i wrote this voice x decision doc. today i update the guide because now it is miles better you can trigger a decision on every input or output frame (roughly every 200ms). you can have a realtime decision engine judging what i’m saying while i’m still saying it!! try realtime action × realtime voice 👇 developers.openai.com/api/do…
3
2
59
3,483
GPT-Live-1.1 offers much better multilingual support (including several new voices), more natural delegation, and improved steerability. Rolling out now to ChatGPT Voice, Desktop, and Dots.
New voices. Better accents. And less "checking..." GPT-Live-1.1 brings these updates to Voice in ChatGPT, Codex, and dots, with new voices in French, German, Hindi, Mexican Spanish, Korean, and Japanese. Rolling out today to all users. Coming soon to the API.
15
2
1
140
6,689
This is very good news, we need interpretability now more than ever
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works already-clean.goodfire.com/
2
1
13
1,298
Echo cancellation is a complicated situation. While gpt-live is quite robust to echo, AEC and AGC are very intertwined and accordingly local AEC is still useful to ensure the model receives a usable signal.
Replying to @juberti
do you think that in the long term, a separate echo cancellation step (neural/linear) still makes sense, or just send raws(/+loopbacks) and let the model figure that out too?
1
1
1
40
2,296
Background voice cancellation can help but also is underparameterized for this situation. If a human might have trouble with an audio task, a small model has no chance.
1
15
1,820
IMO diarization is the wrong direction. It’s putting effort into generating human legible features when the right solution is just training the LLM to handle background noise (ie bitter lesson strikes again).
Replying to @juberti
Can some kind of built-in diarization help with such cases in your opinion?
7
1
46
3,106
Exactly. Orchestration is just a crutch to help the model scale up. Don’t get too attached to it.
The model eats the harness, aka bitter lesson
4
2
53
2,776
Why turn detectors (or any other low-parameter endpointing mechanism) are a dead end
How to kill a voice agent? Just turn on the TV ☠️ We benchmarked 14 voice agents with a TV playing in the background. Task completion fell from 38.7% to 8.6% 🧵/
10
4
3
156
14,054
Responses is great, but you can use session.instructions.append to steer the Live API, and it takes effect immediately. developers.openai.com/api/re…
Model steering now live in the Responses API. Always thought this is where things needed to head. Did the responses API team beat @juberti to adding realtime steering? Surely at some point the APIs will converge.
4
40
3,440
There's indeed a lot going on here, but the real kicker is the em dashes.
This document is going to be an assigned reading in college classes that cover this moment in time, there's a lot happening in a few paragraphs... ahmath.org/statements
1
11
1,066
Updated our Live API/Realtime API documentation to show how to use the new WARP extensions for WebRTC to reduce connection latency to a single RTT, typically well under 100 ms. developers.openai.com/api/do…
14
13
4
168
10,066
I’m somehow reminded of the 19th century debates over vitalism.
5
1,175
ChatGPT Voice can just do things! It now supports plugins, Work mode, and delegation to GPT-6 when more intelligence is needed.
We heard you loud and clear. ChatGPT Voice can now: - Use plugins like your email, calendar, and Slack. - Be powered by GPT-6 Astra, Sol, and Luna. - Be used in ChatGPT Work on web and mobile, so you can create docs, decks, sites, and spreadsheets or tackle complex tasks in the browser, just by talking. Rolling out globally today in the latest version of the app.
9
2
53
3,411
Justin Uberti retweeted
"stochastic parrot" was a mimetically-fit cognitive virus that spread from 2021-2025; it temporarily blinded many gifted people to the nature of AI progress, burning up crucial years in which they could have helped think through the response to the situation.
275
350
170
3,447
761,895