Lawson (@lawsonfulton) header
Lawson (@lawsonfulton) profile picture

Leading Reality View @ Tesla SI

Palo Alto
Joined November 2007
Photo by Lawson (@lawsonfulton)Photo by Lawson (@lawsonfulton)Photo by Lawson (@lawsonfulton)Photo by Lawson (@lawsonfulton)Photo by Lawson (@lawsonfulton)Photo by Lawson (@lawsonfulton)Photo by Lawson (@lawsonfulton)Photo by Lawson (@lawsonfulton)Photo by Lawson (@lawsonfulton)
If you are excited to work on this kind of technology, send me a DM. We are hiring world-class engineers at every level of the stack!
Replying to @BLKMDL3 @wholemars
In case anyone is curious: We’ve had a ~pixel-perfect world model since last year, which is critical for synthetic data generation. The (much) smaller version of that in the car is a conditional mean-flow model, based on the recent sequence of Tesla Universal Vision Encoder features, which are the same ones that are plugged into the 3B encoder model that produces the driving actions. We fork off from these UVE features, then run a separate ~1B diffusion model. This takes in the previous reality-view generation with added noise and then denoises it to the current visualization. It then produces a small video chunk (8 images at 36 fps, at the resolution of the UI screen). This work is done by 1/2 of a TRIP engine every UVE frame at 18 hz and is spread over 4 cycles. Currently, the latency is a bit high due to compute constraints, but we have ways to interleave newer UVE features along the diffusion transformer layers to cut roughly 35% of the latency.
18
35
1
571
22,516
When we say we do end-to-end embodied AI architectures, it's in prod, shipped to millions of cars. The much larger world model is shared between Autopilot and Optimus and is very important for 3D labeling for Optimus. The effort is shared across my team, @lawsonfulton and Yekeun. We're hiring!
Replying to @BLKMDL3 @wholemars
In case anyone is curious: We’ve had a ~pixel-perfect world model since last year, which is critical for synthetic data generation. The (much) smaller version of that in the car is a conditional mean-flow model, based on the recent sequence of Tesla Universal Vision Encoder features, which are the same ones that are plugged into the 3B encoder model that produces the driving actions. We fork off from these UVE features, then run a separate ~1B diffusion model. This takes in the previous reality-view generation with added noise and then denoises it to the current visualization. It then produces a small video chunk (8 images at 36 fps, at the resolution of the UI screen). This work is done by 1/2 of a TRIP engine every UVE frame at 18 hz and is spread over 4 cycles. Currently, the latency is a bit high due to compute constraints, but we have ways to interleave newer UVE features along the diffusion transformer layers to cut roughly 35% of the latency.
20
38
3
406
158,485
Tesla FSD v15 is gta in real life
181
504
57
5,235
341,227
Lawson retweeted
Replying to @BLKMDL3 @wholemars
In case anyone is curious: We’ve had a ~pixel-perfect world model since last year, which is critical for synthetic data generation. The (much) smaller version of that in the car is a conditional mean-flow model, based on the recent sequence of Tesla Universal Vision Encoder features, which are the same ones that are plugged into the 3B encoder model that produces the driving actions. We fork off from these UVE features, then run a separate ~1B diffusion model. This takes in the previous reality-view generation with added noise and then denoises it to the current visualization. It then produces a small video chunk (8 images at 36 fps, at the resolution of the UI screen). This work is done by 1/2 of a TRIP engine every UVE frame at 18 hz and is spread over 4 cycles. Currently, the latency is a bit high due to compute constraints, but we have ways to interleave newer UVE features along the diffusion transformer layers to cut roughly 35% of the latency.
732
1,278
416
17,115
1,471,030
Lawson retweeted
FSD is GTA irl
This is NUTS. It goes around a Huge Branch in the Road v15
1,797
2,025
251
25,253
3,372,212
Produced end-to-end, pixels-in-pixels-out
Rodeo Drive as seen by Tesla Reality View on FSD v15!
329
441
62
6,939
204,717
Lawson retweeted
If you zoom out a little further, you can see our simulation playing on the screen of the creators.
Overheard in Palo Alto: GTA 6’s real-world mode just dropped on Teslas.
229
313
41
4,257
514,155
Lawson retweeted
Replying to @Truthful_ast
Breaking: Elon Musk invents 3rd person mode IRL
2
37
2,330
Look how Tesla FSD V15 displays an arrow in the visualization to indicate where you're turning. And handling a roundabout. I feel like I’m looking at a video game right now lol
237
272
82
4,297
471,091
Fun fact, when I was interviewing with @pduan 4 years ago he said: "One day, I want looking at the viz to feel just like you're looking outside" It didn't feel possible at the time. But here we are!
Overheard in Palo Alto: GTA 6’s real-world mode just dropped on Teslas.
18
51
6
1,049
30,522
I honestly still can't believe we pulled it off. Huge shout out to @johnathan_xie and @TerryWa95074331 on their work to design and deploy a blazing fast real-time architecture.
This is real-time AI video generation (no gaming engine!) using just half of one AI4 TRIP accelerator. Let that sink in …
78
65
7
1,952
41,169
Rest of the team also deserves a huge amount of credit: @Shahanneda @amyxlase @lalShamit @JiahaoLi95
24
23
4
690
24,923
Lawson retweeted
Replying to @jamesdouma
It is mesmerizing to watch. Where this is actually useful is when you want to tap the exact drop-off or parking location on the screen that you want your car to target.
307
299
26
5,643
698,630
Lawson retweeted
This is so unbelievably impressive
'Reality View' in Tesla FSD v15 > Adds a live visualization of the drive that lets you see & control what FSD is planning to do.
58
91
5
3,176
163,009
Lawson retweeted
In FSD V15, the Tesla AI computer re-renders reality with chevrons in the sky for turns like a video game. You can tap any point on the reality revisualizer to make your car go there.
2,102
3,757
770
40,179
3,113,064
Lawson retweeted
This is real-time AI video generation (no gaming engine!) using just half of one AI4 TRIP accelerator. Let that sink in …
The crazier thing is that the right side uses less compute than what the left used to take. Extreme compute efficiency optimization.
1,319
1,858
380
20,918
2,918,404
Lawson retweeted
It’s like GTA
Here are the new FSD visualizations in FSD V15. It continuously streams a view from an arbitrary virtual camera situated above your vehicle. Upcoming navigation turns display directly in the driving visualization. Turn signals are quieter and fade out after a few clicks.
2,012
2,754
280
35,660
4,513,399
Lawson retweeted
FSD 15 VISUALIZATIONS ARE INSANE HOW IS THIS EVEN POSSIBLE
FSD v15 now starting to roll out – upgrading every part of the end-to-end driving network Safety 25x better handling of safety critical cases Tesla Reality View  Adds a live visualization of the drive that lets you see & control what FSD is planning to do Navigation for start & end of the trip now learns end-to-end from fleet route preferences. Later v15 releases will expand this to the entire route. You also should notice better detection of signs, animals, debris, potholes and standing water, plus more natural speed handling in school zones and inclement weather. Lastly, FSD can now slow for speed cameras & Bike Rack Mode lets you enable FSD w/ bike racks or similar hitch accessories
106
168
26
4,215
227,679
Look at how good the visualizations are in Tesla FSD v15! 🤯😍 Absolutely stunning
21
16
4
390
9,346
Lawson retweeted
v15 is 25x safer than v14 across the fleet. That’s hard to notice on any individual drive because most drives are already very safe. The difference only becomes clear when you aggregate millions of miles. All credit goes to the incredibly talented and dedicated @TeslaSI engineers who made this possible. It’s an honor to work alongside them.
FSD v15 now starting to roll out – upgrading every part of the end-to-end driving network Safety 25x better handling of safety critical cases Tesla Reality View  Adds a live visualization of the drive that lets you see & control what FSD is planning to do Navigation for start & end of the trip now learns end-to-end from fleet route preferences. Later v15 releases will expand this to the entire route. You also should notice better detection of signs, animals, debris, potholes and standing water, plus more natural speed handling in school zones and inclement weather. Lastly, FSD can now slow for speed cameras & Bike Rack Mode lets you enable FSD w/ bike racks or similar hitch accessories
232
616
78
6,866
671,669