monitoring frontier ai and the cultural pulse in real time i consult on ai engg stuff and looking for more work in benchmarking, evals, rl envs/training etc.
Joined October 2021
- Tweets48.1K
- Following721
- Followers27.2K
- Likes241K
Pinned Tweet
curious people and people interested in llm evals, benchmarks, high data circles, please wake up. my blog post "SWE-bench Multimodal v2 is not multimodal" is up now.
sankalp.bearblog.dev/swe_mul…
timeline cleanse
(i listen to hvnly often when my brain is fried. was the most random collab between JIHYO (kpop) and jenevieve (city pop, r&b)
JIHYO'S OPENING VERSE FOR 'HVNLY'
"I just don't know why I think of you when I close my eyes
How you love to stay on my mind
Swear I said goodbye, something to a hundred times
Musta been my favorite lie
He must be an angel, I'm lovestruck if you can't tell
I think I've found a reason to believe I'm under his spell
I wish somebody warned us, when did he get this gorgeous?
Now I'm running 'round in circles, brainstorming how to solve this"
can i say something without everyone getting mad... i often see some mechanistic interpretability experiments here and people getting excited about them without really understanding what's going on and sometimes it feels like mech int is like astrology for ai people lol
implications:
more cybersec people needed or atleast aura++ (easier to understand for market)
more attacker swarms, more proactive/defender swarms inference time compute++ (memory can rebound but can market understand this?)
these quote tweets suddenly feel more visceral now. need to sign apology form to both anthropic and openai if i criticized the holding back of models.
Introducing Project Glasswing: an urgent initiative to help secure the world’s most critical software.
It’s powered by our newest frontier model, Claude Mythos Preview, which can find software vulnerabilities better than all but the most skilled humans.
anthropic.com/glasswing
sankalp retweeted
Team work makes the dream work <3
We wanted to do something a bit different than using LLM's to generate all the pieces from exploit, writeup and art. We decided on a human created comic to offset the LLM usage in our release, that tells the tale with some exaggerated scenarios inspired from a few movies.
If you liked the art (or want your own exploit story/comic..) check out her work at ibiscraft.art
Do you know who illustrated the comic on calif.io/php2shell?
Her name is Stephanie Cortes. And her husband is the hacker who discovered php2shell: @lcfr_hax.
This artist-hacker couple has the best story, stay tuned!
quick coverage of posts and relevant articles on the CVEs posted yesterday
trackingsingularity.com/e/ph…
openclaw -> cringe
hermes -> nice greek aura
grok bot -> cute name
dots -> meh
claude duet ->
1B series A is crazy
We’ve spent the past year developing new primitives for GPUs. We believe this enables new algorithms to change how machines learn, open new markets, and dramatically reshape AI’s cost structure.
We have raised a $1 billion Series A, co-led by Index and Lux.
big thanks to @xeophon @omouamoua for helping me along the way and @PrimeIntellect for the infra and credits
sankalp retweeted
A good heuristic for building multimodal evals is running it without the images
Well, turns out that SWE-bench multimodal v2 doesn’t really need them 🙃
Great detective work by @dejavucoder!
curious people and people interested in llm evals, benchmarks, high data circles, please wake up. my blog post "SWE-bench Multimodal v2 is not multimodal" is up now.
sankalp.bearblog.dev/swe_mul…
something interesting and different than your regular speeding up kernel or optimizing inference server mill of "RSI bench" tasks
Recursive self-improvement is being hyped to the moon and back. I wanted to know how good frontier models actually are at a less-studied piece of it: curating pretraining data.
So I built PretrainGym at the @PrimeIntellect RL Residency 🧵
justin-chae.org/blog/pretrai…
jesse it's time to cook
We’ve acquired Orchestra Bio, a platform that helps biopharma teams coordinate the complex process of drug development.
Together, we’ll build on Rosalind, our AI models and tools for life sciences research, to streamline those operations and help new treatments reach patients sooner.
Welcome @ashokaraj_ and team!
openai.com/index/openai-to-a…
