
🇮🇩; pre-MEI hire @scale_ai, always in faith and fury
SF
Joined March 2019
- Tweets841
- Following374
- Followers953
- Likes3.4K
Pinned Tweet
great american models:
to distill a mockingbird
kv cacher in the rye
the great gptsby
4
11
1,895
Quality Data Is All You Need
Inkling didn't need looping nor 7-8T parameters
what you need is actual quality data
the Chinese are still, 3 years of my yelling later, with double-digit educated youth unemployment, unwilling to pay good money for data collection. I guess they'll get what they deserve then.
1
6
538
Brandon Handoko retweeted
good model behavior requires care for craft from the top
craft is hard at bigger companies with so many cooks in the kitchen, where product execs obsess over a/b test metrics and research execs over eval numbers. it requires a ton of work & political capital to make & advocate for something that feels invisible when good, and push back against efforts that ruin it -- and eventually people burn out
so then you end up with bad model behavior, and maybe a few people complain internally but then you launch anyway and then twitter blows up and then oh no we need a code red on model behavior but none of the structural problems go away
Brandon Handoko retweeted
Had a great conversation with @yacinelearning about @ScaleAILabs’ FineART, which just surpassed 1 million downloads!
We covered why data diversity matters more than sheer volume, how subtask-level annotations help robots learn complex, multi-step behaviors, and what it will take to build more capable, general-purpose robots! 🤖
often time the solution to better modelling performance is simply in the data
and robotics is no different
in this 1h30 interview I sat down with @jadechoghari from scale ai to discuss their absolutely bonker way of improving data by literally labelling fine grained sub tasks!!
Brandon Handoko retweeted
Created a "benchmark benchmark": a frequency distribution of which benchmarks were reported by the frontier labs in 2026: claude.ai/artifact/22cBPJt6A…
Some observations:
- HLE, SWE-Bench Pro, DeepSWE, and Terminal-Bench saw universal adoption
- Scale AI is behind two of those four (SWE-Bench Pro, HLE w/ CAIS), plus MCP-Atlas (#9 benchmark)
- Of the frontier labs, OpenAI is the most prolific benchmark producer (SWE-bench Verified, BrowseComp, GDPval); people don't seem to mind evaluating on a benchmark made by a competitor
- Anthropic has just one on the chart (BioMysteryBench) that basically nobody uses 🤔
- In academia, strong showings from Princeton (SWE-bench, CharXiv), Stanford (Terminal-Bench), Berkeley (CyberGym, Agents' Last Exam), and NYU (GPQA)
- DeepSWE saw incredible adoption, from first appearing in June to all 12 labs reporting it
- Labs cycled through four versions of Terminal-Bench (2.0 → 4.0) in a single year
Amusing meta experiment: putting things in my slack statuses have generated far and away more curious, genuine engagement than any super visible @ channel ping has
3
6
567
Lotta folks trying to tackle human behavior simulation these days that feels transactional / marginally beneficial to the actual human.
@onecaseman + @ludo_antonov are sitting on a product that reverses that with gorgeous network effects && properties that force multiply and grow alongside the actual you -- one of the handful of demos that has stuck w me this year. Huge for us all! :-)
You shouldn't need the best network to get the best insights.
SuperMe turns operators' work into profiles that work through your problem. Now in group chats, Claude and ChatGPT.
@ludo_antonov and I raised a $20M Series A led by @kirstenagreen from @ForerunnerVC with participation from @GreylockVC.
Read more about the changes here: superme.ai/blog/series-a
1
6
493
sf ball knowers breakfast burrito ranking:
mj’s cafe > il canto cafe > breakfast little >>> rest
3
350
Brandon Handoko retweeted
the whole idea of having your ai agent vibe code me a mediocre “homecooked” ui feels disrespectful…
just bc my mom can technically code now, doesn’t mean she’s suddenly become a great product person (or is even just remotely interested in becoming one).
making product is an art. it takes a GREAT GREAT deal of discipline because simplicity is a function of deleting more than you add even when EVERYONE will pull you towards adding more and more. ai 100x this pressure because building is so easy.
i’ve barely seen any ai products with a STRONG, polarizing POV.
if eng still is your bottle neck you’re not spending enough time thinking, designing, and deleting. then you end up with an experience that can do so many things, your users have to figure out how to use them. some sf tech bros will do so but most people are simply not interested. they expect you to do that work for them.
liberate yourself from the pressure of having to do everything for anyone or you will fail leaving behind a massive trail of slop.
Brandon Handoko retweeted
Gemini 4 Argon and Muse Wonder on HLE-Diamond.
Gemini 4 Argon competes with GPT-6.1 Sol and Opus 5.5, but uses ~5x more output tokens than GPT-6 models.
I’ve long been a fan of smaller conferences like CoLM: more targeted, more privy to meandering + intimate conversations, less hectic etc
Case in point: @natolambert now knows how much of my career he set up at Scale through his work!
28
920
Brandon Handoko retweeted
I don’t like to take down ambitious companies but there’s a fine line and at this point I’ll just say I would bet cumulative revenue of Brett’s companies never reach cumulative peak valuation of them.
Maybe by a huge margin.
I truly do not know how people keep funding these things and Hark has got to be some sign of market top.
Brett Adcock is one of humanoid robotics’ biggest hype men. @Figure_robot is valued at $39B; @Forbes says he's worth $23.4B. Nearly two years ago, he said he saw a path to 100,000 robots across two customers in four years. How far has Figure gotten? Our profile 🤖: forbes.com/sites/annatong/20…
Brandon Handoko retweeted
The strange thing about VCs investing in competitors is how normal they pretend it is.
When a VC takes a board seat, they aren't merely making an investment. They become a fiduciary. They gain access to the company's strategy, performance, weaknesses, and plans. And they accept a duty to put the company's interests ahead of their own.
Then their firm invests in a competitor. We're supposed to believe the conflict disappears because everyone involved is professional. But how many firms actually prevent partners from seeing how competing companies are marked? How many require them to recuse themselves from portfolio reviews? How many have real information barriers, rather than assurances that no one will misuse what they know?
A conflict doesn’t have to produce obvious misconduct to be a conflict. Simply knowing that one company is accelerating while another is struggling influences where a firm puts its attention, capital, recruiting help, and introductions.
The board member’s job is to bet on the company. If their firm needs to bet on its competitor too, perhaps they shouldn’t have taken the board seat.
(A sidenote: I’m not talking about situations where a portfolio company pivots and begins competing with another company in the portfolio. I’m talking about knowingly investing in competitors. And yes, founders sometimes claim that every company in their broad space is a competitor, when very often it isn’t.)
Brandon Handoko retweeted
GPT-6.1 Sol and Sonnet 5.5 on HLE-Diamond.
GPT-6.1 Sol matches Opus 5.5 while being much cheaper in token usage.
Brandon Handoko retweeted
SWE Atlas has been accepted to NeurIPS 2026! We built the benchmark to test whether agents can work on coding tasks beyond simple issues, with professional SWE rigor!
We will continue to track model progress on this and have some exciting news coming soon!
Paper - arxiv.org/pdf/2605.08366
Brandon Handoko retweeted
Career advice I did not expect to give:
go work for the “data vendor” 😭
You may not be building the frontier model, but you get exposed to problems across a bunch of frontier labs: audio, multimodal, coding, multilingual, safety, physical AI, RL environments, and whatever new thing appeared this week.
As data becomes half the game and RL-env startups casually raise millions, this has been an absurdly good place to learn.
Getting to publish open research at the same time = 🫶
Brandon Handoko retweeted
That's more like it.
Astra is strictly dominant on the *cleaned-up* HLE.
We are releasing HLE-Diamond, a refined subset of Humanity’s Last Exam (HLE), following a year-long process of cleaning and refinement with input from various research communities.
lastexam.ai/blog/hle-diamond
w/ @ScaleAILabs
Brandon Handoko retweeted
Replying to @brndnwll
Looking forward to humanitys-last-exam-last-final-final-revision-final.bench
Awesome to work on this with @longphann_, @MantasMazeika96, @oliverzhang42, @hendrycks – CAIS is always on the case 💎💎💎💎
We are releasing HLE-Diamond, a refined subset of Humanity’s Last Exam (HLE), following a year-long process of cleaning and refinement with input from various research communities.
lastexam.ai/blog/hle-diamond
w/ @ScaleAILabs
1
7
681
































