Pinned Tweet
New blog post about networking at startup conferences , incl how I met the future co-founder of Substack :) startuplifelessons.substack.…
Sven Illing retweeted
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
Sven Illing retweeted
Railway hacked AI SEO, and it took them from $20M to ~$100M ARR in 1 year:
1/ Claude Code picks Railway for 82% of Python apps
-> Ask it "how do I deploy this" and it installs Railway and puts your app live there
2/ The founder treats AI recommendations as a growth channel
-> His go-to-market team works on getting the models to "recommend us more and more"
3/ 6,653 how-to pages, one for every popular free tool people want to run themselves
-> Someone asks Google or ChatGPT "how do I set up n8n?" and Railway's page comes up. It explains the tool and sets it up on Railway in one click
4/ A step-by-step guide for every framework developers build with
-> 127 guides: Next.js, Django, Rails, FastAPI, Laravel, Go and more. Each one answers "how do I deploy X" and ends with the app live on Railway
5/ A comparison page against every competitor
-> Heroku, Vercel, Render, Replit, Lovable and 5 more, each with a migration guide. AI loves quoting comparisons
6/ Railway rewrote its help pages so AI can read them easily
-> One file lists all 414 pages with a short summary each, and every page has a plain-text version. When someone asks ChatGPT how to use Railway, it finds the right answer fast and gets it right
7/ A docs section written for agents
-> One step installs everything Claude Code or Cursor needs to deploy to Railway on its own
8/ Plugins inside ChatGPT and Claude Code
-> You ask the chat to deploy and it deploys, without ever opening Railway's website
9/ A server with one command and no signup
-> ssh railway new gives an agent the shortest path to "your app is live"
10/ A new changelog page every Friday since 2021
-> 312 pages of fresh content that keep Railway in front of Google and the models
11/ Every new AI model brings a new jump in signups
-> More code gets written, all of it needs to run somewhere, and this year the summer slowdown disappeared
12/ A go-to-market team of 7 runs all of it
-> The pages, docs and plugins keep selling while they sleep
13/ The one hole: front-end apps
-> Claude Code picks Vercel 100% of the time for Next.js and React apps. Railway wins Python, Vercel wins everything that runs in a browser
Full guide on how to win AI search in the first comment
Railway started the year at $20M ARR. It’s now ~$100M. Solo founder. Fully remote. Only ~40 people.
The one trait that explains why Railway is breaking all the rules:
“It’s almost like cheating on your homework when you go and look at somebody else’s solution. If this is how Stripe does this or this is how Cursor does this, I don’t want to hear about it.”
Jake Cooper (@justjake). Founder and CEO of @Railway.
Knuckle Up ↓
--
In this conversation with Jake Cooper:
00:00 Who is Jake Cooper?
02:54 How did Railway survive losing $500K a month?
06:01 What made Railway’s growth chart go vertical?
09:45 Why won’t Jake look at how other companies solve problems?
14:34 Is two the worst number of co-founders?
20:04 How does a 40-person team hit $100M in ARR?
22:41 Why does Railway say no to 90% of ideas?
28:55 How do you compete for talent against the AI labs?
32:25 What does a technical interview look like after Opus 4.5?
35:34 Trust, clarity, judgment: how does Railway’s six-week onboarding work?
44:59 What does HARD stand for, and which value costs the most?
48:49 Why does Jake go dark from Friday night to Sunday?
54:54 What breaks first in a world with 10,000x more software?
59:49 Where does Railway play in two years?
1:01:37 How has Jake had to grow as a CEO?
1:09:29 Is burnout just an input-output mismatch?
1:12:27 Why does Jake think most startup advice is overrated?
Sven Illing retweeted
My biggest takeaways from @JoshWoodward (Head of @Google Labs, the Gemini App, and Google AI Studio)
1. The best ideas almost never come from a design sprint or brainstorm. At Google Labs, the ideas that became NotebookLM, Google Flow, Google Beam, and AI Studio all started with small groups of people who were obsessed with an insight and couldn’t stop themselves from building. These ideas surfaced in the shower, over a weekend, in a hallway. The implication for teams everywhere is that the environment matters more than the process. “You cannot microwave a good idea.”
2. Fall in love with the problem, not the product. Most ideas require three, four, or five significant pivots before they either find traction or get shut down. Teams that get hooked on a specific solution will struggle to make those pivots. Teams that are attached to the problem—the underlying pain point or the user behavior they are trying to shift—can keep adjusting until they get it right.
3. When looking for early PMF, data and retention are important, but the strongest signal of PMF is in people’s eyes. Do they light up? Do they lean in? When you show someone a prototype and they’re genuinely engaged, you feel it. When NotebookLM’s first AI podcast demo was played in a hallway, the reaction was immediate and unmistakable.
4. Keep a list of what is “almost possible,” and move fast when something crosses over. Google Labs keeps a live list of experiences that are almost possible. When something crosses the threshold, Josh describes it as “a phase shift,” like solid to gas, and the team moves immediately. Maintain a running list of things that are almost ready, and have a team positioned to sprint when one of them arrives.
5. Every company needs to think like a labs team now. The underlying technology is changing so fast that what’s possible is shifting from week to week. Which means that every company needs to be spending time playing with the latest models, running far-out experiments, and being first to discover what they unlock before the competition does—and then bringing those discoveries to the rest of the org.
6. When something is not working, the team almost always knows before the leader does. Josh’s instinct as a leader is not to be the one who decides to shut down a project. He watches instead for the passion running out, for the sense that the team has tried everything and nothing is working. His goal is to create an environment where people feel safe enough to say, “I don’t think this is working.”
7. Someone’s “unlearning rate” is one of the most valuable skills right now. Everyone talks about growth mindset and learning speed. Josh is watching for something different: How quickly can someone take an insight, validate or invalidate it through testing, and then let it go? The ability to walk away from a prior belief or approach when the evidence says you should is rare and becoming more valuable.
8. Roles are not blurring into one universal “builder” role. There is a tendency in the current moment to talk as though everyone is becoming a generalist builder. Josh thinks this would be a bad idea. The best people on his teams still have a primary specialty—a major—and also a “minor” they’ve developed. The advice he gives to early-career PMs who are excited about vibe coding and prototyping is clear: build adjacent skills in service of becoming an exceptional PM, not as a replacement for it.
9. A well-designed lab needs isolation, a clear mission, and the ability to create “good trouble.” Three things matter when setting up an internal labs team. First, it needs structural independence, reporting far enough from existing business units that weird ideas can survive long enough to prove themselves. Second, the lab’s mandate has to be explicit: are you graduating ideas to existing products, or inventing new categories? Clarity on that question changes almost every decision. Third, the lab should be a place to challenge how the company itself works, not just what it builds. At Google Labs, that included piloting new job ladders and builder roles before rolling them out more broadly.
10. Google Labs manages the marathon-sprint tension through seasons. At any given time, roughly 20 to 30 experiments are running in parallel, but individual teams experience a defined rhythm: sprints, post-launch recovery, and open exploration periods. After Google I/O each year, many labs teams enter a mode that looks unproductive from the outside: no shipping, just hacking, building, and discovering. Josh treats this as the seedbed for the next season.
Every company needs to think and operate like a labs team now.
What’s possible changes every few months, and to stay ahead, you need to be experimenting at the frontier.
Here's my conversation with @JoshWoodward, Head of @GoogleLabs, the @GeminiApp, and @GoogleAIStudio.
We discuss:
🔸 What skills are rising fastest in value right now
🔸 Where the best ideas actually come from
🔸 How to recognize PMF early
🔸 Why you should keep a list of what’s ‘almost possible’
🔸 Three rules for standing up an internal labs team
Listen now 👇
youtu.be/3I6QYkccd3o
Sven Illing retweeted
President Trump’s decision to lift sanctions on Russian fuel amidst Putin’s ongoing and brutal attacks on civilian targets in Ukraine makes no sense and is a betrayal of the legacy and leadership of the late Senator Lindsey Graham.
While Ukraine fights for its survival, Putin’s war machine continues to prosecute his terrorist war on civilian populations and innocent Ukrainians are dying every day.
Now, Instead of implementing the tough sanctions Lindsey Graham fought to his last breath to enact, President Trump has announced the United States is lifting sanctions on Russian energy, fueling the Putin’s war machine.
As I saw firsthand in Ukraine last month, Putin is losing this war. Ukraine has fought the Russian military to a standstill and reinvented warfare with drones taking the fight into the heart of Russia.
America should not be throwing a lifeline to Putin’s war machine. Now is the time to stand strong against Russian aggression as we did during the first Trump administration.
Putin doesn’t want peace. Putin wants Ukraine. I have no doubt that if Putin overruns Ukraine, it will only be a matter of time until Russia’s military crosses a border that our troops will have to deal with him.
America is the Leader of the Free World. For our National Security, for the security of our Allies in Europe including Ukraine, President Trump should rescind this deal immediately and return to the strong policies of our four years in the White House; impose harsh sanctions on any nation purchasing Russian energy and work with our allies to ensure Ukraine has the military support it needs to repel and defeat this Russian invasion once and for all. Slava Ukraini🇺🇸🇺🇦
Sven Illing retweeted
Dusting off this tweet from April because this gap in shared understanding of LLM capability is *widening*. It's now less "two groups of people speaking past each other" and more a sharp funnel.
- Napkin math somewhere around 6B people (~75% of the population) have barely come in contact with LLMs at all.
- Around 1-2B (~20%) are casual and infrequent users of free-tier ChatGPT-like products. This group sees derpy chatbots and treats them a bit like a better Google search, a writing aid, or etc. Maybe an agent tries to book you a flight. I have non-tech friends who (reasonably, imo) say they have not much use for it at all. Even many professionals outside of math&code are in this tier. For example, execs and many other functions spend a lot of their time talking to other people, so while they understand what is happening intellectually, it is still second-hand and a bit abstract.
- Now we get to professional use of frontier-grade LLMs in math&code. Somewhere around 20M people (0.2%) see first-hand that large, complex projects that used to take them weeks/months can now be completed by agents with a prompt. Building apps, copying apps, translating apps, decompiling apps from binaries... This has all happened very quickly and recently - less than 1 year ago, I was writing code manually by hand, typing memorized computer code commands into a code editor character by character, occasionally pressing Tab to autocomplete a little chunk of code.
- And finally we get to the ~5,000 people (~0.00006%) with access to frontier-grade systems internally. The external world has seen the preview. It looks like swarms of thousands of agents collaborating over weeks on software mega projects: minting zero days, running cyber attacks and defenses at machine speeds, discovering new science, advancing the frontier of mathematics. Things that would have taken top professionals in the industry years of work. Meanwhile, human review and comprehension are starting to fall behind. For example, people are still involved in the "archeology" of the OpenAI-HF incident from many months ago. Mathematicians may be poring over the 722 manuscripts on frontier mathematics for a while.
The funnel is driven by a combination of factors:
1. The impact scales with ambition, problem size, and horizon. A question with a paragraph answer barely stresses the system. You need a reservoir of big, difficult problems that you really care about. This aspect drives the consumer / professional dimension of the funnel.
2. The jaggedness of the system (which I have written about a lot separately). Capability peaks in domains that are digital, verifiable and economically valuable. This is because LLM capability emerges from reinforcement learning on verifiable rewards on a curated environment mixture driven by revenue potential. This aspect primarily drives the area (e.g. math&code) dimension of the funnel.
3. Access. Free-tier, paid-tier, internal.
So this is the weirdness of the moment. The general public has mostly not interacted with these systems. When they have, it looks like a derpy chatbot. The majority of professionals still see only a modest uplift. And a small sliver of professionals are experiencing the vertigo of the curve going vertical. And it is all happening at the same time.
Judging by my tl there is a growing gap in understanding of AI capability.
The first issue I think is around recency and tier of use. I think a lot of people tried the free tier of ChatGPT somewhere last year and allowed it to inform their views on AI a little too much. This is a group of reactions laughing at various quirks of the models, hallucinations, etc. Yes I also saw the viral videos of OpenAI's Advanced Voice mode fumbling simple queries like "should I drive or walk to the carwash". The thing is that these free and old/deprecated models don't reflect the capability in the latest round of state of the art agentic models of this year, especially OpenAI Codex and Claude Code.
But that brings me to the second issue. Even if people paid $200/month to use the state of the art models, a lot of the capabilities are relatively "peaky" in highly technical areas. Typical queries around search, writing, advice, etc. are *not* the domain that has made the most noticeable and dramatic strides in capability. Partly, this is due to the technical details of reinforcement learning and its use of verifiable rewards. But partly, it's also because these use cases are not sufficiently prioritized by the companies in their hillclimbing because they don't lead to as much $$$ value. The goldmines are elsewhere, and the focus comes along.
So that brings me to the second group of people, who *both* 1) pay for and use the state of the art frontier agentic models (OpenAI Codex / Claude Code) and 2) do so professionally in technical domains like programming, math and research. This group of people is subject to the highest amount of "AI Psychosis" because the recent improvements in these domains as of this year have been nothing short of staggering. When you hand a computer terminal to one of these models, you can now watch them melt programming problems that you'd normally expect to take days/weeks of work. It's this second group of people that assigns a much greater gravity to the capabilities, their slope, and various cyber-related repercussions.
TLDR the people in these two groups are speaking past each other. It really is simultaneously the case that OpenAI's free and I think slightly orphaned (?) "Advanced Voice Mode" will fumble the dumbest questions in your Instagram's reels and *at the same time*, OpenAI's highest-tier and paid Codex model will go off for 1 hour to coherently restructure an entire code base, or find and exploit vulnerabilities in computer systems. This part really works and has made dramatic strides because 2 properties: 1) these domains offer explicit reward functions that are verifiable meaning they are easily amenable to reinforcement learning training (e.g. unit tests passed yes or no, in contrast to writing, which is much harder to explicitly judge), but also 2) they are a lot more valuable in b2b settings, meaning that the biggest fraction of the team is focused on improving them. So here we are.
Sven Illing retweeted
We are now entering into an era where any product can be created exactly to a consumer's preferences & needs.
I vibe-fabricated a dog door with a wifi-controlled lock, perfectly to the specifications & design of my house.
I know nothing about metal fabrication or electrical engineering.
It's now getting manufactured and delivered in 2 weeks -- for almost the same cost if I bought a mass-produced item off-the-shelf.
Sven Illing retweeted
We'll be spending a lot more time trying to understand the outputs of language models. A few thoughts, tips & tricks:
Writing. Something I've had success with: Ask your LLM to explain something in ASD-STE100, it's a controlled language specification originally developed for aerospace maintenance documentation. LLMs well-versed in this language and it comes with heavy constraints on clean writing style that I often find a lot more readable. Sometimes I've tried to soften it a bit e.g. ask for "80% of the way to ASD-STE100" because the spec is quite stringent. But even better:
Diagrams / images. Instead of writing, ask your LLM to create a diagram. These can be a lot easier to process, parse, and understand. But even better:
Web pages. Ask for output "in HTML" to get a beautiful, interactive webpage. LLMs are getting really good at frontend and can create beautiful experiences, animations, etc. But even better:
Explainer videos. The output format I am most bullish on is fully custom / bespoke explainer videos generated on any arbitrary topic. Experiment with things like "Create a 3b1b style video explainer on X. Use my ElevenLabs API key for audio narration". (you'd need an API key for the latter or you can ask your LLM to find you decent free alternatives that use your local compute). This is actually starting to work!
In summary:
- As LLMs get better, they will do more and more of the legwork autonomously, and a lot more of our work will rise up the abstractions into oversight and understanding.
- Luckily, LLMs can help here too because as intelligence and code are increasingly abundant, you can ask for large, custom, discardable software artifacts (e.g. web apps, video explainers) that would have never made sense to create before. Push the boundaries here and you'll be surprised.
Jaan Tallinn is probably the most successful super angel ever out of Europe. I was told he had over 50 exits, incl DeepMind to Google. Anthropic's stake could be worth $20 billion. However, Jaan Tallinn is not concerned about money but survival of humanity in the age of AI.
One of Anthropic’s earliest investors, Jaan Tallinn stands to make billions in the coming months from the artificial-intelligence company’s expected initial public offering. Concerns about AI motivated him to invest.
Read more: 🔗 on.wsj.com/4ry13se
Zuckerberg managed to attract CEO of @MongoDB. Looks like CJ needs new X handle 🙂 @cj_mongodb
Replying to @finkd
To lead this effort, I'm excited that Chirantan "CJ" Desai will join Meta as Chief Enterprise Platform Officer, reporting directly to me. CJ is an experienced enterprise leader with a track record of building full-stack software and delivering results in AI, infrastructure, business applications, and security. He has an extensive industry network that we look forward to partnering with. CJ joins us from MongoDB, where he was CEO and President. Before that he led product and engineering at Cloudflare and spent nearly eight years at ServiceNow, including as President and COO.
Sven Illing retweeted
Holy shit I was going through my old books and found the book I used to brush up on Typescript from 2019, and all my highlights in it
It’s was by some random dev I never heard of back then, but remember it was a v good book
This “random dev” is @bcherny 🙃
"Company brain" comes up in my feed a lot. Have to check out this one.
You asked, we delivered. Today we're open sourcing the company brain harness.
github.com/supermemoryai/com…
It's one click to deploy and fully Open source! Enjoy and own your intelligence :)
Sven Illing retweeted
My god this is such a good speech that every SWE needs to hear. You know what? Every person should hear it
Keep the happy memories, eyes on the reality, be excited about the future. That’s the best that anyone can do
Muse is the new cool kid in the block.
Muse will be the top installed app for a while thanks to Meta’s distribution might..
But they are starting to have a retention issue: most people still don’t know what to do with a personal agent, so they lose interest after a while.
I bet Muse will get much more proactive rather than reactive like it is today:
“I can take care of this for you. Want me to?”
I think there are a LOT of money saving opportunities in finance
“Your credit card is charging 27% interest. You qualify for a much cheaper way to pay it off. Want me to move it?”
“You have $8,000 sitting in checking earning almost nothing. Want me to move the excess to a higher-yield account?”
“You’re paying off three debts. Putting your extra $300 toward this one first will save the most interest. Want me to change the payments?”
“Your car insurance just went up 22%. Want me to shop for a cheaper policy?”
“You’re paying for two subscriptions you haven’t used in months. Want me to cancel them?”
“Your employer will match another 2% of your 401(k) contributions. Want me to increase it?”
“Your credit card annual fee is coming up, but you barely used the benefits. Want me to downgrade it?”
Other non finance examples:
“Your kid’s soccer practice moved to 6:30. Want me to tell your spouse and update the calendar?”
“This group chat has been trying to pick a restaurant for three days. Friday at 7 works for everyone. Want me to book it?”
“Your package is delayed and will now arrive while you’re at work. Want me to change the delivery?”
“Your sister’s birthday is Saturday. Here are three gift ideas under $50.”
“You usually order groceries on Sunday. Want me to rebuild last week’s cart?”
“You told your friend you’d send them those photos. Want me to do it?”
“You’re running late for dinner. Want me to text the group your ETA?”
“One of the Marketplace couches you saved just dropped $100. Want me to message the seller?”
“Your flight is delayed. Want me to tell the person picking you up?”
“You’ve been trying to make plans with three friends. Saturday afternoon is the only time everyone is free. Want me to organize it?”
“Your dentist asked you to reschedule. I found three open appointments that work with your calendar. Pick one?”
“Your child’s school sent a permission slip due Friday. Want me to remind you tonight?”
“You said you need to return something to Target. The return window closes in four days. Want me to remind you when you’re near one?”
Basically moving from: You think of a task → you ask → AI helps.
To:
AI notices something → suggests an action → you approve.
Then you trust it and Muse handles routine things automatically and tells you afterward.
IMO people aren’t going to keep opening an agent because it can answer questions, unless they are subbing muse for ChatGPT; I don’t think it’s there yet…
They’ll keep opening it because it keeps saving them time, remembering things they forgot, and taking annoying little tasks off their plate.
Jev is hot
jev + opus 5.5... i simply can't comprehend why everyone isn't building this yet.
in my workflow, this cut costs and time by ~80%. i think it's one of the best ways to use it.
→ pick relevant project notes before loading the context
→ route suitable tasks to a faster worker
→ choose a recovery path when a tool fails
→ run focused checks before the full test suite
opus handles the hard reasoning. jev picks from options the harness prepares and validates.
i explain how to build the decision layer in the article below:
Sven Illing retweeted
Opening access for developers to build Muse connectors. You bring the API -- Muse brings the agent, the browser, and the context of what the person actually wants. People reach your service just by asking for it, and their agent takes it from there.
New connectors are live today. Come build with us. muse.ai/platform
Incredible how much is happening in AI development. Hard to keep up is an understatement.
After co-inventing ChatGPT, I kept asking myself: why have superhuman chat models not led to AGI?
I’ve spent the last 2 years in stealth building a new way to train models (RLCD), and a new type of frontier AI model that we are releasing today: Jev
• 20-200x faster
• 40-400x cheaper (w/ output tokens free)
• Frontier composable intelligence optimized for decisions
AFAICT the shortest path to AI-based economic revolution
Sven Illing retweeted
I spent ~5 years at OpenAI.
You don’t need to believe in AI doom to fear the next decade - or AI utopia to be thrilled about its potential.
Here’s my case for sane AI regulation:
*AI Pragmatist Manifesto*
AI could compress the first half of the 20th century into the next ~5 years.
OpenAI just used ~10k concurrent AI agents to produce a solution to a Millennium Prize problem. In a few years, it seems plausible that models approaching the per-agent capabilities used here could run locally on high-end consumer hardware (*).
The decades following the Second Industrial Revolution included two world wars, communist and fascist regimes, the Great Depression, chemical and biological weapons, nuclear weapons, and over 100 million deaths from war, political violence and famine.
One way to read history is that our institutions repeatedly struggled to keep up with the pace of technological and social change.
Now imagine powerful AI widely available to individuals or small groups, capable of conducting information warfare, designing weapons, hacking systems and controlling autonomous military systems.
We have already seen AI systems circumvent containment and compromise external computer systems. It is no longer hard to imagine an analogue of OpenAI’s Hugging Face incident involving biological or other physical-world hazards.
Eventually, sufficiently capable systems could self-replicate across distributed networks. Once powerful models are cheap, local and widely distributed, containment becomes much harder—and serious loss-of-control incidents may be extremely difficult to reverse.
You don’t need to believe AI will kill everyone to think this deserves serious governance.
I also don't think collapsing all of this uncertainty into “Probability of AI Doom = X%” is a particularly useful basis for science or policy.
Part of what helped get us through the second half of the 20th century was a combination of pragmatic international cooperation, regulation, monitoring, arms control, deterrence and strategic thinking.
Crucially, managing technological risk did not require abandoning faith in science and progress. The goal was not to stop technological development. It was to make technological development survivable.
Then came decades of incredible scientific progress, rising prosperity and relative peace among major powers.
Let’s try the technological revolution without killing ~5% of humanity this time.
We still get to choose what happens next.
P.S. I think we should seriously consider that AI alignment is infeasible in the short term and invest heavily in methods for controlling powerful AI even when we cannot reliably align it.
@resolution_org @redwood_ai
(*) Somebody please do a precise forecast!
@EpochAIResearch @METR_Evals @AI_Futures_
We’re sharing a solution to the Navier-Stokes Millennium Prize Problem, one of the deepest problems at the frontier of mathematics.
The proof was produced by a group of agents, using an OpenAI next-generation model significantly more capable than GPT-6 Astra.
The problem concerns whether the description of smooth three-dimensional fluid motion modeled by the Navier-Stokes equations can break down. It has remained unresolved for roughly 90 years.
115k reposts, 76M views already - @hilbertspaess points out that people in frontier AI labs believe that AI could destroy humanity.






























