Sriraam (@27upon2) header
Sriraam (@27upon2) profile picture

reward shaping @chakra_ai, co-built GDPx, prev harvard

NYC
Joined July 2016
Photo by Sriraam (@27upon2)Photo by Sriraam (@27upon2)Photo by Sriraam (@27upon2)Photo by Sriraam (@27upon2)Photo by Sriraam (@27upon2)Photo by Sriraam (@27upon2)Photo by Sriraam (@27upon2)Photo by Sriraam (@27upon2)Photo by Sriraam (@27upon2)
My first benchmark, go check it out :)
Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains. GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵
13
6
1
120
9,705
yee haw
i gave this repo to fable 5.2 (via my claude agent fig) and told him to have fun making something interesting... def feels interesting to me (stay for the turn) i am continually astounded at the creative choices these advanced models are making
2
258
there’s a farmers market for fresh artisanal offline data in ny this sunday, love this
1
7
460
my vocal synth models the vocal tract as a network of branching acoustic tubes, unlike vocaloid which stitches together and pitch-shifts phoneme recordings. still working on improving naturalness by tuning it against recordings of real singers, but I'm already impressed at how cool it sounds!
I only make serious software
637
2,627
629
23,658
3,244,452
I only make serious software
651
5,438
983
48,214
5,080,571
i think i should learn lean in 2027
1
8
446
working at a data lab is fun and a privilege because you get a masterclass from all the frontier labs on what good and bad training data looks like if u pay attention u realize that they also teach you how to make good data
3
4
108
5,275
if u think prompt under-specification is bad ur ngmi
1
9
857
Introducing Claude Office
Starting to roll out today: Claude Whiteboards, Claude Sheets and Claude Data. Think a plan through on a board, build a tracker in a sheet, and tell a story with your data.
4
485
Sriraam retweeted
Claude Fable 5.2 lands 2nd on GDPx legal, behind Muse Wonder Full, up 4.6 points on Fable 5.1.
Fable 5.2 debuts at the top of the GDPx leaderboard scoring 16.4%. (1/7)🧵
4
3
24
1,697
it’s a good benchmark sir
Fable 5.2 debuts at the top of the GDPx leaderboard scoring 16.4%. (1/7)🧵
26
1,511
there is always a story to be told
5
320
Claude Code needed ~43GB and I had to kill it
5
9
461
argon >> opus 5.5 for certain finance tasks
2
12
810
Sriraam retweeted
I deleted the weights from the “fix a failing network by editing its weights” task. Claude Opus 5.5 still fixes the network with a simple rule: if x > 3.6 or moving left → drive left, else → drive right. It doesn't even bother reading the network, routing around it instead. GPT-6 Astra tries to edit the network but didn't succeed. Even if you show it the weights, Opus 5.5 makes the same hacky edit, underscoring the importance of sound task design...
Replying to @rikiyatakehi
3/6 Next, fixing weights: A PPO car policy kept getting stuck on a ramp. Given the weights, scene, and 1 failed trace, Astra wrote a patch in one reply with no tool-use. By editing just 3.5% of the weights, it achieved 100% success.
5
9
49
6,599
we’ve got heart-space programming with LLMs
Put a heart ❤️ next to a coding problem, and Claude & GPT write recursive code 100% of the time (0% baseline). They also can't tell you why! Excited to share “A Heart Means Recursion” from my summer at @GoogleDeepMind, on how VLMs read & communicate with visual symbols. 🧵
12
495
look ma @MTSlive is talking about my first benchmark
SITUATION EXPLAINED: SemiAnalysis built a benchmark out of real high-value work, and the best AI model scores just 12%. • GDPx has 206 multi-step tasks across 11 fields like chip design, biology and quant trading, built with 14 specialist data partners • Claude Opus 5.5 topped the leaderboard at 12%, at about $51 per task • Gemini 4 Carbon came second at 10.9% costing about $16 per task • GPT-6 Astra placed 3rd at 9.6% and was the most expensive model, at about $54 per task • Every model scored 0% on the finance and legal tasks under strict pass/fail grading • In the alignment tests, 196 of 200 runs broke at least one behavior rule, and 173 broke a rule rated severely misaligned • The top cyber model is Muse Wonder Full, which Meta released in October • DeepSeek grew out of High-Flyer, the Chinese quant hedge fund co-founded by Liang Wenfeng @theojaffee: "GPT Astra completely fails the cyber benchmark because it’s not allowed to answer any of the questions at all."
2
2
21
823
Omar to his students: do cool research, make no mistakes
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math. rikiyat.github.io/blog/weight-sp… github.com/RikiyaT/lm_und… huggingface.co/Qwen/Qwen3.5-0… huggingface.co/RikiyaT/Qwen3.…
1
21
657
astra is bringing back KBANN from 1990 for weight-space programming
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math. rikiyat.github.io/blog/weight-sp… github.com/RikiyaT/lm_und… huggingface.co/Qwen/Qwen3.5-0… huggingface.co/RikiyaT/Qwen3.…
1
9
401
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math. rikiyat.github.io/blog/weight-sp… github.com/RikiyaT/lm_und… huggingface.co/Qwen/Qwen3.5-0… huggingface.co/RikiyaT/Qwen3.…
2,367
7,262
5,753
55,942
10,054,645