Christian Vestergaard retweeted
very interesting, they pruned 2/3 of the experts during mid training then did logit distillation from a larger inkling model
i would expect this to be a big part of why they get so much intelligence for the limited number of parameters
Some of you thought that Inkling 2 Small was too big at 594B parameters. We heard you: Inkling 2 Mini is a looped transformer with just 26B parameters, small enough to run on your laptop, powerful enough for agentic work with a score of 20.1 on AA Index.
thinkingmachines.ai/news/ink…
Christian Vestergaard retweeted
what i’ve come to understand and hold true in my head but am always revisiting…
1. we don’t have enough raw resources (memory, cpu, power, data centers) - this won’t go away for perhaps up to 6 to 10 years
2. There is minimal to no moat for the prop labs; except selling poison and cure. They will make money selling expensive expensive cyber tokens in response to the security problems created by AI
3. AI fundamentally has changed work and there’s no going back. The box is open and businesses need fewer humans to run a businesses.
4. AI is an economic munition. If a country depends upon AI (and they will) and their access is suspended then a country will be economically crippled by such sanctions.
5. Openweight models are typically three months behind prop labs. They are a critical counter to $ant.
6. Countries must develop their own sovereign open weight models which codify their culture/customs to protect/encode their nations value structures to protect against cultural homogenization.
7. Distillation is competition and fair game. How it is done is should be the focus of conversation (not “if” it’s allowed) - ie distillation currently happens through theft/credit card fraud - that isn’t good!
Though tough thoughts am pondering about:
8. Training on pre-existing IP is perhaps not theft of IP. The same way that a human reading a book, seeing an idea, learning concepts and then remixing that concept isn’t a copyright violation. Mathematics isn’t IP/copyright theft.
Christian Vestergaard retweeted
this is it. 1000% it.
people who think something is too late are usually not founders. one could argue that book business was done and amazon appeared to change the game. blockbuster was the king when netflix was the cd delivering peasant til the empire was overthrown.
again and again when the non founders think the game is over, the founders appear and change the game.
it's an infinity war if you choose to believe it, and the game is always on.
Scott Wu on the nihilism creeping into tech: people saying it's "too late" to build an independent company because the labs will just do everything.
His take: those people aren't founders. Founders are rationally optimistic with zero evidence they'll succeed.
If you think coding agents look like this forever, you're wrong. There are 10 more generations of this product to build.
Christian Vestergaard retweeted
The evals are not ok
New NeurIPS paper: Hidden Measurement Error in LLM Pipelines Distorts Annotation, Evaluation, and Benchmarking.
Evals drive deployment, safety standards, and research. Yet we ignore noise from prompt, judge, agent harness etc, & so underestimate error in benchmarking & annotation, & it gets worse with more data.
I'll get to what this means for CIs & how to fix, but first let's talk about how this makes benchmark hacking easier 🧵
Christian Vestergaard retweeted
To make something *truly* great, you always have to put in more time and effort than you want.
The good news is most don’t, so if you do, it’s all the more likely it’ll stand out.
Christian Vestergaard retweeted
We believe most interaction with AI will happen through speech. But models still talk like machines.
This is a data problem.
Today we're launching @agenticdataco (YC F26): real-world speech data + audio-native RL environments so speech models sound and act human.
Christian Vestergaard retweeted
Today, we're introducing Duo.
Duo is a full-duplex conversational AI. It listens while it speaks and hears your voice directly instead of a transcript, so it reacts to pauses, a quick "mm-hm" and interruptions as they happen.
Don't be so cool you can't appreciate cool stuff. Bring back some of that earnest childhood wonder, some of that pure unadulterated WOW! Retire your cynicism.
“This is the most exciting time to be into computers”
When I interviewed @dhh at our Omarchy Shenzhen meetup 2 weeks ago what stood out was his absolute enthusiasm - everyone in the room could feel it
😎
Christian Vestergaard retweeted
Replying to @khuxsal
Not at all. 4 years is very short. Social media glorifies shipping speed but real, durable ideas all take a decade or more. Very few quick wins that aren't just flash in the pan. Dig in deep and expect the good ideas to be full time work for a long, long time.
If you hang out with happy, optimistic, and ambitious people, it'll rub off too. That's why pruning your social media algorithms is so key. Don't psyop yourself into a depressive hole.
every breakthrough in my life has come from a bout of obsession...
1-2 months where i lock myself in the office, eyes glued to the screen, forgetting to eat
i've never found balance to work for me - in my eyes balance = maintenance, growth comes in bouts of obsession
i wish i could consciously trigger this obsession for anything i wish, but it seems like it's a force that takes over when it needs to
Christian Vestergaard retweeted
Our OpenYap Turn model takes #1 for end-of-turn on @sesame's TurnBench
Trained on 1,984rs of real conversation between friends and family.
Read the writeup: theagenticdatacompany.com/re…
Christian Vestergaard retweeted
if you value intelligence above all other human qualities, you’re gonna have a bad time
Christian Vestergaard retweeted
Feeling more optimistic about European tech than ever.
Spent the week in building alongside the startups at ninethreequarters.com in copenhagen, and what a cocktail of talent and ambition
Christian Vestergaard retweeted
If you want to get ahead, just be easy to work with. Show up on time. Do what you said you'd do. Bring solutions, not problems. Never create drama. Be responsive. Be emotionally consistent. Be kind. People will always want to support someone who just makes their life easier.


























