Building stuff to help people build better stuff
Australia
Joined February 2025
- Tweets758
- Following186
- Followers55
- Likes8.8K
Ben retweeted
Calling it now.
AI is going to usher in a whole new era of software development. I’m calling it Software as a Service.
People will want to use these tools but they’ll realize they don’t want to spend the time to maintain and update them over time.
This will create a whole new category of businesses and entrepreneurs that can take on that task for businesses that would rather work on their core products.
AI is changing everything. This is a whole new paradigm of thinking about the way we interact with software.
I wonder what the linguistic implications of this could be. If the primary interface for computers becomes language, that will be what most people spend most of their time talking to.
At what point does the interface become the programming language?
The two worst kind of engineers right now:
"AI is useless, my expertise is irreplaceable."
"AI is omnipotent, my expertise is obsolete, I must reinvent myself."
Clanker: There's other agents running. We'll need to wait until they complete to restart the service.
Me: make it hot reloadable.
Clanker: yessir
It's that easy, bros.
Ben retweeted
Everyone forgets that Claude also flew.
It’s the same when the PR gets merged,
or the UI fails and people say
they knew it was a mistake, that everybody
said agents would never work. That Theo was
old enough to know better. But anything
worth shipping is worth shipping badly.
Like those charts of three hundred merges a day
while the warnings piled up, the code
burning so extravagantly those nights that
anyone could tell you they would never last.
I believe Claude was not failing as he fell,
but just coming to the end of his triumph.
Code review is still the bottleneck because agents lack your team's shared understanding of the software. Without that, there's only so much you can trust their judgment.
Trust is the real bottleneck. Giving agents that shared understanding is the next big challenge.
Ben retweeted
This is why you need your own evals on your own codebase.
Does this actually help?
Does it actually matter?
Or does having more files actually degrade performance?
Prompt of the day:
Create a pre-commit hook that prevents files over 5,000 tokens. Tokens = bytes/4.
Run only on staged files. Ignore files which are not authored by agents, like lockfiles or codegen.
Provide explicit instructions on how to split files in the error message.
The simplest investment I ever made in my codebase. So good.
I don't think we realise the cognitive load skills puts on agents. A typical code review skill asks an agent to keep dozens of things in mind before it's even started reviewing anything.
Well defined tasks are useful, but I am pretty sure there's a better way.
Your model solved a maths problem we've been working on for a century? Cool, I guess...
Can it one shot videogames tho?
The smartest code model in the world would not be good at 3d stuff unless it was trained specifically for it.
A lot of the labs recently got a big pile of 3d training data and an RL environment to improve 3d skills.
Take a look at the jump in 3d from Fable 5.1 to Opus 5.5. It's at least 4x better at 3d despite being a 5% better model at best.
Clearly Meta didn't go hard on 3d for this release. Doesn't mean the model is bad.
3dmaxxing is a different type of benchmaxxing, don't overthink it.
I'll take some fault here by sharing all the Fishslop demos. The 3d stuff is fun and cool as hell, but it's not a good representation of a model's real world code capability.
Software's peculiarity is that it's never really done. What you've learned and the decisions you have made are what make it better. You can clone a snapshot of what those discoveries produced, but you'll still struggle to catch a team that's been building something for a decade.
I could probably learn how to make my own shoes. I could probably build a machine to make my own shoes. I might be able to use an agent to get the shoes made and on my feet faster.
I am almost definitely going to keep going to footlocker because I like shoes but do not want to be a shoemaker.
Ben retweeted
> I don't understand the code the LLM wrote for my CRUD app
> I know, extremely high-end techniques like formal verification and dependent types will fix me
I can see a future where professional yak shaving becomes its own thing for engineers outside of the traditional lifecycle of software dev. People continuously honing the feedback and discovery cycles of the machine.
Lots of people were already kinda doing this anyway.
Ben retweeted
I don't think you're in a position to talk about benchmarks when you're rejecting submissions.
Are the people putting out 2500 prs a month just doing tiny little micro-optimisation PRs all the time? Is any of the work actually good?
The feature factory thing feels achieveable now, but I feel like just maintaining it becomes the job at that stage.
I think people are getting a whiff of "let them eat cake" smell from the flight booking demo. Most people's air travel isn't for work, it's for holidays. Which, most people I know enjoy doing.
There's value in taking something inaccessible to most people, like an executive assistant, the favoured tarpit of the decade, but democratising the solution is only useful if you're solving problems your users actually have. The second half to that post they didn't do was figuring out what regular people actually want taken off their plate.
Rich people are probably keeping their EAs.