Shamit (@lalShamit) profile picture

@TeslaSI Prev: Nova Models at @amazon AGI

Joined November 2013
Shamit retweeted
Replying to @BLKMDL3 @wholemars
In case anyone is curious: We’ve had a ~pixel-perfect world model since last year, which is critical for synthetic data generation. The (much) smaller version of that in the car is a conditional mean-flow model, based on the recent sequence of Tesla Universal Vision Encoder features, which are the same ones that are plugged into the 3B encoder model that produces the driving actions. We fork off from these UVE features, then run a separate ~1B diffusion model. This takes in the previous reality-view generation with added noise and then denoises it to the current visualization. It then produces a small video chunk (8 images at 36 fps, at the resolution of the UI screen). This work is done by 1/2 of a TRIP engine every UVE frame at 18 hz and is spread over 4 cycles. Currently, the latency is a bit high due to compute constraints, but we have ways to interleave newer UVE features along the diffusion transformer layers to cut roughly 35% of the latency.
742
1,294
421
17,374
1,508,913
Grateful to have contributed to Reality View alongside an incredible team.
This is real-time AI video generation (no gaming engine!) using just half of one AI4 TRIP accelerator. Let that sink in …
11
12
2
493
9,120
Shamit retweeted
New BlaGPT best: 3.1153 val loss, with Latent Lattice. One small recurrent latent (width 256) runs across layers 2-7. At each layer it steers q/k/v and the MLP through FiLM. Paper: arxiv.org/abs/2611.21608 Code + results: github.com/erogol/BlaGPT
6
18
762
TESLA FSD V15 IS NOW ROLLING OUT! Check out the self-driving UI! 🤯🤯 It literally rendering your entire surroundings. THIS IS CRAZY!
105
310
80
4,225
341,082
Shamit retweeted
FSD v15 now starting to roll out – upgrading every part of the end-to-end driving network Safety 25x better handling of safety critical cases Tesla Reality View  Adds a live visualization of the drive that lets you see & control what FSD is planning to do Navigation for start & end of the trip now learns end-to-end from fleet route preferences. Later v15 releases will expand this to the entire route. You also should notice better detection of signs, animals, debris, potholes and standing water, plus more natural speed handling in school zones and inclement weather. Lastly, FSD can now slow for speed cameras & Bike Rack Mode lets you enable FSD w/ bike racks or similar hitch accessories
3,038
5,965
3,176
44,308
8,213,271
Shamit retweeted
Replying to @roger_kappler
Yeah. The new user interface will re-render reality in a mind-bending way. Hard to describe, but it’s awesome.
370
487
176
8,218
988,254
Shamit retweeted
Replying to @elonmusk @wholemars
The new interface is amazing
233
157
45
5,137
491,932
Language models process each token through a single residual stream (blue arrows): reading / writing memory and applying MLPs at each layer, and finally decoding the next token. Can we scale up this single stream computation? Introducing Latent Lattice Language Models (LLLMs): augmenting this vertical stream with a horizontal axis of computation—the Lattice of Latents (orange arrows). RNN 🤝 Transformers 🧵
19
65
8
424
33,184
Tokenization & Generation power Large Models. But are they really separate? Tokenization=Generation under strong observability UNITE: An end-to-end training framework where one shared Generative Encoder (GE) performs both token. & latent denoising Paper: arxiv.org/abs/2603.22283
4
78
8
421
70,182
Similar thought. Next-token prediction feels statistical: perplexity / shannon-entropy minimization. But creativity / science may require: finding compact generative structures, then exploring in that space. Closer to algorithmic complexity? More Kolmogorov than Shannon.
Being good at next word prediction is the opposite of what we want for creativity, for scientific breakthroughs.
3
6
69
9,337
Compression is the heart of intelligence From Occam to Kolmogorov—shorter programs=smarter representations Meet KARL: Kolmogorov-Approximating Representation Learning. Given an image, token budget T & target quality 𝜖 —KARL finds the smallest t≤T to reconstruct it within 𝜖🧵
14
60
8
373
64,698
Current vision systems use fixed-length representations for all images. In contrast, human intelligence or LLMs (eg: OpenAI o1) adjust compute budgets based on the input. Since different images demand diff. processing & memory, how can we enable vision systems to be adaptive ? 🧵
10
64
11
475
93,736
Our new #CVPR2021 paper with @mihirp98, @ishitamed, @AdamWHarley, and Katerina Fragkiadaki is out on arXiv!
CoCoNets: Continuous Contrastive 3D Scene Representations pdf: arxiv.org/pdf/2104.03851.pdf abs: arxiv.org/abs/2104.03851 project page: mihirp1998.github.io/project…
1
8
Check out this #CVPR21 paper from our lab, addressing the voxel resolution issues from our previous unsupervised 3D feature learning approaches. Now we have continuous (subvoxel) features, which outperform voxel-contrastive and point-contrastive learning!
CoCoNets: Continuous Contrastive 3D Scene Representations pdf: arxiv.org/pdf/2104.03851.pdf abs: arxiv.org/abs/2104.03851 project page: mihirp1998.github.io/project…
8
25
Shamit retweeted
Disentangling 3D Prototypical Networks For Few-Shot Concept Learning pdf: arxiv.org/pdf/2011.03367.pdf abs: arxiv.org/abs/2011.03367 project page: mihirp1998.github.io/project…
43
8
165
Hello @ICC - how are you coping your end? #Wimbledon #CWC19Final
886
15,695
1,789
71,826
Tie in ODI finals Aus vs WI MCG 1984 - B & H World Series Cup Eng vs Aus Lord's 2005 - NatWest Series Eng vs NZ Lord's 2019 - World Cup #ENGvNZ #CWC19Final
2
48
3
559
FOUR Productivity FEYNMAN- strategies: i) Stop trying to know-it-all. ii) Don't worry about what others are thinking. iii) Don't think about what you want to be, but what you want to do. iv) Have a sense of humor and talk honestly.
16
1,655
111
4,051
Learn: 1. linear algebra well (e.g. matrix math) 2. calculus to an ok level (not advanced stuff) 3. prob. theory and stats to a good level 4. theoretical computer science basics 5. to code well in Python and ok in C++ Then read and implement ML papers and *play* with stuff! :-)
Replying to @ShaneLegg
@ShaneLegg Hey Shane I’m currently 17 from London England and am very passionate about AI, also learning about in-depth human needs. What would be the 5 pieces of advice and tips you would give to a young person like me?
18
349
40
1,251
Texter is a latex symbol retriever. It is developed using @tensorflowjs which allows an offline version and a much faster prediction time then Detexify. DEMO : zaidalyafeai.github.io/texte… CODE : github.com/zaidalyafeai/zaid…
4
17
52