Pinned Tweet
If you are excited to work on this kind of technology, send me a DM.
We are hiring world-class engineers at every level of the stack!
Replying to @BLKMDL3 @wholemars
In case anyone is curious:
We’ve had a ~pixel-perfect world model since last year, which is critical for synthetic data generation.
The (much) smaller version of that in the car is a conditional mean-flow model, based on the recent sequence of Tesla Universal Vision Encoder features, which are the same ones that are plugged into the 3B encoder model that produces the driving actions.
We fork off from these UVE features, then run a separate ~1B diffusion model. This takes in the previous reality-view generation with added noise and then denoises it to the current visualization.
It then produces a small video chunk (8 images at 36 fps, at the resolution of the UI screen). This work is done by 1/2 of a TRIP engine every UVE frame at 18 hz and is spread over 4 cycles.
Currently, the latency is a bit high due to compute constraints, but we have ways to interleave newer UVE features along the diffusion transformer layers to cut roughly 35% of the latency.
Lawson retweeted
When we say we do end-to-end embodied AI architectures, it's in prod, shipped to millions of cars.
The much larger world model is shared between Autopilot and Optimus and is very important for 3D labeling for Optimus. The effort is shared across my team, @lawsonfulton and Yekeun. We're hiring!
Replying to @BLKMDL3 @wholemars
In case anyone is curious:
We’ve had a ~pixel-perfect world model since last year, which is critical for synthetic data generation.
The (much) smaller version of that in the car is a conditional mean-flow model, based on the recent sequence of Tesla Universal Vision Encoder features, which are the same ones that are plugged into the 3B encoder model that produces the driving actions.
We fork off from these UVE features, then run a separate ~1B diffusion model. This takes in the previous reality-view generation with added noise and then denoises it to the current visualization.
It then produces a small video chunk (8 images at 36 fps, at the resolution of the UI screen). This work is done by 1/2 of a TRIP engine every UVE frame at 18 hz and is spread over 4 cycles.
Currently, the latency is a bit high due to compute constraints, but we have ways to interleave newer UVE features along the diffusion transformer layers to cut roughly 35% of the latency.
Replying to @BLKMDL3 @wholemars
In case anyone is curious:
We’ve had a ~pixel-perfect world model since last year, which is critical for synthetic data generation.
The (much) smaller version of that in the car is a conditional mean-flow model, based on the recent sequence of Tesla Universal Vision Encoder features, which are the same ones that are plugged into the 3B encoder model that produces the driving actions.
We fork off from these UVE features, then run a separate ~1B diffusion model. This takes in the previous reality-view generation with added noise and then denoises it to the current visualization.
It then produces a small video chunk (8 images at 36 fps, at the resolution of the UI screen). This work is done by 1/2 of a TRIP engine every UVE frame at 18 hz and is spread over 4 cycles.
Currently, the latency is a bit high due to compute constraints, but we have ways to interleave newer UVE features along the diffusion transformer layers to cut roughly 35% of the latency.
Lawson retweeted
Look how Tesla FSD V15 displays an arrow in the visualization to indicate where you're turning. And handling a roundabout.
I feel like I’m looking at a video game right now lol
Fun fact, when I was interviewing with @pduan 4 years ago he said:
"One day, I want looking at the viz to feel just like you're looking outside"
It didn't feel possible at the time. But here we are!
I honestly still can't believe we pulled it off.
Huge shout out to @johnathan_xie and @TerryWa95074331 on their work to design and deploy a blazing fast real-time architecture.
Rest of the team also deserves a huge amount of credit:
@Shahanneda @amyxlase @lalShamit @JiahaoLi95
FSD 15 VISUALIZATIONS ARE INSANE
HOW IS THIS EVEN POSSIBLE
FSD v15 now starting to roll out – upgrading every part of the end-to-end driving network
Safety
25x better handling of safety critical cases
Tesla Reality View
Adds a live visualization of the drive that lets you see & control what FSD is planning to do
Navigation for start & end of the trip now learns end-to-end from fleet route preferences. Later v15 releases will expand this to the entire route.
You also should notice better detection of signs, animals, debris, potholes and standing water, plus more natural speed handling in school zones and inclement weather.
Lastly, FSD can now slow for speed cameras & Bike Rack Mode lets you enable FSD w/ bike racks or similar hitch accessories
v15 is 25x safer than v14 across the fleet.
That’s hard to notice on any individual drive because most drives are already very safe. The difference only becomes clear when you aggregate millions of miles.
All credit goes to the incredibly talented and dedicated @TeslaSI engineers who made this possible. It’s an honor to work alongside them.
FSD v15 now starting to roll out – upgrading every part of the end-to-end driving network
Safety
25x better handling of safety critical cases
Tesla Reality View
Adds a live visualization of the drive that lets you see & control what FSD is planning to do
Navigation for start & end of the trip now learns end-to-end from fleet route preferences. Later v15 releases will expand this to the entire route.
You also should notice better detection of signs, animals, debris, potholes and standing water, plus more natural speed handling in school zones and inclement weather.
Lastly, FSD can now slow for speed cameras & Bike Rack Mode lets you enable FSD w/ bike racks or similar hitch accessories



































