harb (@harbware) header
harb (@harbware) profile picture

engineer light cardio enjoyer bio/ml nerd

Joined May 2021
Photo by harb (@harbware)Photo by harb (@harbware)Photo by harb (@harbware)Photo by harb (@harbware)Photo by harb (@harbware)Photo by harb (@harbware)Photo by harb (@harbware)Photo by harb (@harbware)Photo by harb (@harbware)Photo by harb (@harbware)
The day you stop challenging yourself is the day you stop evolving. Surround yourself with people who challenge you.
8
23
2
173
13,502
men will literally walk out on anything in 30 seconds flat if they feel the heat around the corner instead of going to therapy
111
1,589
120
19,597
612,470
I have a major result to share. I’ve found conclusive forensic evidence that the top 4 Chinese AI labs—DeepSeek, Moonshot, MiniMax and Ant Group—have built their newest models on NVIDIA’s most advanced Blackwell chips, which are barred from export to China. This sounds insane, but I discovered that almost every AI model carries a hidden signature of the GPU that trained it. When a model is first set up, the starting values of its weights are laid out in a pattern that depends on the exact GPU. Each combination of GPU and seed number produces a unique pattern—a sort of fingerprint of the hardware used to create the model. Traces of that pattern survive training into the published model. Again, this evidence isn’t based on any insider information or leaking. It’s based on a forensic analysis of the model weights released by the companies themselves. I searched more than six trillion combinations of chip layout and seed number and matched them to open-weights models. In doing so, I found conclusive evidence that six Chinese AI models—including DeepSeek-V4-Pro, Kimi K3, and MiniMax-M3—were initialized on NVIDIA Blackwell systems. Four of the models match Grace Blackwell, the GB200 and GB300 parts in NVIDIA’s flagship liquid-cooled rack-scale systems. I then verified these results on real hardware. Running unmodified PyTorch with the seeds I’d recovered, a Blackwell B300 reproduced DeepSeek-V4-Flash’s untouched starting weights bit for bit, and a rented GB200 reproduced Kimi K3’s to a correlation of 0.999995. A Hopper H100 reproduced none of DeepSeek-V4-Flash’s, and changing the seed by one broke the match. (DeepSeek’s seed is 953049583 by the way.) The method also was verified to match 24 known open-weights models to the hardware that developers said they used. It is exceedingly unlikely that these results are a coincidence. Across six trillion combinations, the best accidental match should reach only about 8 sigma (i.e., eight standard deviations above what random chance would produce). The real matches range from 49 to 1,385 sigma, and even the weakest single test hits 20. These were data-center-scale training runs. With some analysis of release schedules and model details, I estimate that the largest, Kimi K3 and DeepSeek-V4-Pro, used 2,000 to 3,000 Blackwell GPUs each. Clusters of this size cost on the order of $150 to $200 million each to buy, and across these labs alone would add up to more than half a billion dollars of export-controlled GPUs, not counting experimental compute or otherwise. It’s quite likely that we see multi-billion dollar expenditure on export-controlled GPUs. I am highly confident in this result, and it has been independently verified by senior researchers at OpenAI and Anthropic. You can read the full report at unemploymentlabs.com/checkpo…. There is much more I’ve identified about these models through other forms of checkpoint forensics like this, and will have more to share soon.
1,811
6,393
2,180
41,077
6,610,502
harb retweeted
Replying to @AdamWolnikowski
I remember pouring you a glass at Jensen’s house during the 2019 Nvidia intern party. I made the clutch move of ordering us pizza at 1am…
12
29
3
4,311
148,877
# on shortification of "learning" There are a lot of videos on YouTube/TikTok etc. that give the appearance of education, but if you look closely they are really just entertainment. This is very convenient for everyone involved : the people watching enjoy thinking they are learning (but actually they are just having fun). The people creating this content also enjoy it because fun has a much larger audience, fame and revenue. But as far as learning goes, this is a trap. This content is an epsilon away from watching the Bachelorette. It's like snacking on those "Garden Veggie Straws", which feel like you're eating healthy vegetables until you look at the ingredients. Learning is not supposed to be fun. It doesn't have to be actively not fun either, but the primary feeling should be that of effort. It should look a lot less like that "10 minute full body" workout from your local digital media creator and a lot more like a serious session at the gym. You want the mental equivalent of sweating. It's not that the quickie doesn't do anything, it's just that it is wildly suboptimal if you actually care to learn. I find it helpful to explicitly declare your intent up front as a sharp, binary variable in your mind. If you are consuming content: are you trying to be entertained or are you trying to learn? And if you are creating content: are you trying to entertain or are you trying to teach? You'll go down a different path in each case. Attempts to seek the stuff in between actually clamp to zero. So for those who actually want to learn. Unless you are trying to learn something narrow and specific, close those tabs with quick blog posts. Close those tabs of "Learn XYZ in 10 minutes". Consider the opportunity cost of snacking and seek the meal - the textbooks, docs, papers, manuals, longform. Allocate a 4 hour window. Don't just read, take notes, re-read, re-phrase, process, manipulate, learn. And for those actually trying to educate, please consider writing/recording longform, designed for someone to get "sweaty", especially in today's era of quantity over quality. Give someone a real workout. This is what I aspire to in my own educational work too. My audience will decrease. The ones that remain might not even like it. But at least we'll learn something.
658
3,619
837
18,756
2,297,441
this stuff is so tiresome. jp morgan lead new york city yacht club to two successive cup victories, rockefeller raced trotters through central park, ellison made sailboats fly you all are falling victim to the pastiche of martyrdom without any of the glory. get over yourselves
Best founders I backed had no hobbies. Only one obsession — the company.
79
185
27
3,561
224,883
Paul Graham: You'll become like whoever you work with
3
28
3
377
12,684
Most people who dream of a quiet life without work are just exhausted from work they hate. The reason you get bored on vacation after a week is that you are meant to move, pursue something exciting, and engage your body and mind in something useful. Do whatever it takes to find work that energizes you. It may take years, but it will have the most meaningful effect on your quality of life.
277
608
79
7,069
309,122
harb retweeted
80% of personal websites only have a single blog post from two years ago where the author mentions they want to write more
191
306
112
7,758
281,621
harb retweeted
>send potentially life changing risky text >anxiously wait for her to text back >pace around the room >phone buzzes >instantly check it like a trained dog >it’s an email from chick-fil-a telling me they updated their privacy policy
68
2,322
58
68,460
630,090
kawhi finding out those shoppers optimum points are still good
58
1,590
65
23,302
429,565
local model? oh you mean my wife
47
819
27
14,384
337,934
Good morning
34
2,811
123
78,272
846,864
harb retweeted
counterpoint
there's one thing i always bet on. it's that everyone sucks. the absolute lack of talent in the world will shock you. what is obvious to you hasn't been done because people suck. it is a constant. no one has any skills whatsoever. you are on a different planet entirely
33
718
13
7,552
189,673
if you talk to every investor up >80% this year at institutional scale they all have kind of the same view in private ai will not lead to super intelligent utopia, it’ll plateau far before then, but it will allow everyone to wirehead themselves on fake companionship, labor, etc
58
138
30
4,027
368,176
Replying to @Brady_H
2
4
2
77
7,793
Typical winter day in Canada. Don’t let that central Canada propaganda fool you 😂
76
60
58
1,605
94,954
I've never felt this much behind as a programmer. The profession is being dramatically refactored as the bits contributed by the programmer are increasingly sparse and between. I have a sense that I could be 10X more powerful if I just properly string together what has become available over the last ~year and a failure to claim the boost feels decidedly like skill issue. There's a new programmable layer of abstraction to master (in addition to the usual layers below) involving agents, subagents, their prompts, contexts, memory, modes, permissions, tools, plugins, skills, hooks, MCP, LSP, slash commands, workflows, IDE integrations, and a need to build an all-encompassing mental model for strengths and pitfalls of fundamentally stochastic, fallible, unintelligible and changing entities suddenly intermingled with what used to be good old fashioned engineering. Clearly some powerful alien tool was handed around except it comes with no manual and everyone has to figure out how to hold it and operate it, while the resulting magnitude 9 earthquake is rocking the profession. Roll up your sleeves to not fall behind.
2,554
7,302
2,906
55,180
17,109,318
My morning routine when I was at my fastest: -Wake up. -Down some water. -Bathroom. -Out the door for a run. It took 10 minutes. Routines are great. But don’t major in the minors. The main thing is showing up for the things and people you care about most.
19
14
3
260
49,462
Uncmaxxing > Looksmaxxing Brother, go to bed early Setup your tax structure Organize your closet Buy an expensive recliner Tell the waiter “we hated it” as you hand over your empty plate
58
457
11
11,128
228,324