Co-founder / CTO @ https://teapawt.org/t.co/ENhthcEt5P
San Francisco, CA
Joined January 2008
- Tweets682
- Following722
- Followers531
- Likes3.5K
Human review is declining. AI code review scaled more than 6x since Jan 2025. Defect rates did not meaningfully move.
AI reviewer bots now produce 28% of PR review comments across our dataset, up from 5% in January 2025. Today, 96% of companies use at least one AI code review tool, with the median using two.
That growth is happening as review demand outpaces human capacity. Over the last nine months, merged PR volume increased 53%, while the reviewer pool grew only 10%. At the same time, the share of PRs merging with no human review rose from 13% to 19%.
The obvious question is whether that changes software quality.
So far, we do not see evidence in either direction for equally sized PRs. PRs merged with no human review were about as likely to be associated with a bug within seven days as comparable human-reviewed PRs. Mostly AI-written PRs were also about as likely to be associated with a bug as mostly human-written PRs of the same size. The one exception is very large changes: in the largest tenth of PRs, mostly AI-written changes were associated with bugs about twice as often as comparable human-written ones. We cannot yet tell whether that reflects the AI itself or the kind of large change engineers still write by hand.
My interpretation: AI review may be helping teams absorb growing code volume without an obvious decline in near-term quality. But we also do not yet see evidence that more automated review reduces defects, and this analysis does not capture the broader benefits traditionally associated with human review.
One factor had a much stronger relationship with bugs: PR size. Larger PRs had a higher probability of being associated with a bug.
I suspect practices such as smaller changes, stronger testing, type safety, and focused bug bashes remain your most important quality levers.
Fable produces fewer defects for just $0.08 more per shipped AI line than Opus.
That’s what we observed with Fable 5 versus Opus 4.7 even though Fable handled a more complex task mix (94% vs. 83%). Composer was also a clear cost outlier on the Pareto curve.
Span makes this analysis possible by connecting agent trajectories to merged code and downstream defects. Early results but still interesting. What is one defect worth to you and your customers?
We’ll repeat this for GPT-5.6 Sol and control for confounders for Composer if there's interest.
Does anyone actually have any evidence that model routers work in terms of $/merged AI line? Switching models mid session incurs a potentially massive cache penalty. Choosing a dumber model than is required for a task increases rework and chance of escaped defect.
Henry Liu retweeted
Exclusive: Span, a startup building a new-look 'developer intelligence' platform for the AI code era, has raised $25M.
Led by Lattice's former head of product J Zac Stein, Span already helps companies like Braze, Intercom, Ramp, Vanta and Writer keep track of their engineering.
Thank @ctsmithiii for writing about Span's code detector!
This really highlights the blind spot eng leaders have right now. AI is change the way we build faster than we can measure it.
coderlegion.com/5422
Henry Liu retweeted
I’m excited to announce the launch of @Span_Platform’s AI Code Detector! 🚀
We all know how transformative AI coding assistants have been, but it's still hard to know what's AI vs. not & what impact it's ultimately having on quality, velocity, and security. Now you can with Span—powered by our machine learning model, span-detect-1, which detects AI-generated code with an industry-best 95% accuracy.
We can’t wait to see how this helps engineering leaders who want to lead with hard data, not hype. Try it out today—completely free.
Henry Liu retweeted
The middle manager is the biggest culprit of the "quiet quitting" SWE epidemic.
They have 0 incentive to fire. The entire job is bargaining for more headcount so they can get promoted.
They'll say "we are understaffed, we need more people" no matter how little they do.
1/4
Henry Liu retweeted
the flight to AI for customer support is such a miss for early-stage companies
the most important thing your company does is make a thing for people to use and you want to let that feedback go to a response bot?
absolutely not
Love the energy of this tweet.
Ok - Gemini day 2 recap: things people like, things we gotta fix. Keep your feedback coming. We're reading it all.
THINGS PEOPLE LIKE (♥️♥️♥️)
- Writing style
- Creativity for helping you find the right words/ideas
- Speed of responses
- Not hitting usage caps
- Online connectivity
- Images in responses
- Integration with tools (Gmail, Maps)
THINGS TO FIX ASAP (we're on it!)
- Refusals = top priority
- Mobile app not available: still rolling out; opening to LATAM, Africa, Asia Pacific, Canada on Monday, Europe ASAP
- Coding: y'all want an interpreter. ack.
- Preachy guardrails: some laughable examples, we'll fix
- Assistant tasks: calendar, reminders, routines extensions
- Android app: conversational overlay input submit
First 2 months on us, let's build together: gemini.google.com/advanced
LET'S GO 🚀🚀
Henry Liu retweeted
Sam should get his job back, the board should be replaced by founders and investors who have skin in the game, the nonprofit should be converted to a C-corp, and Elon should get shares for putting in the first $40M+. In other words, undo all the shenanigans.
Henry Liu retweeted
Today, we officially introduce EchoMark, unveiling an unprecedented solution for information protection. EchoMark automatically embeds invisible forensic watermarks in documents and email, personalized for each recipient. Featured in @natebek’s @GeekWire bit.ly/3tcU3aj
Henry Liu retweeted
$500 rent in SF would create a vibrant city
Dean Preston and his NIMBY friends systematically block housing, making already rich homeowners more wealthy while displacing middle/low income residents, and excluding immigrants
Vote him out
DumpDean.org
Henry Liu retweeted
People don't understand Bryan Johnson. So I'm going to engage in the terrible Twitter thread format to explain his work.
1/ He sold Venmo/Braintree to PayPal and made FU money. "what's the point of making FU money if you don't say FU?" He's also ex-Mormon. Explains tweet below
Henry Liu retweeted
Invented the accelerator. Probably the most successful female investor in Silicon Valley history. Wrote one of the best books about startups. But notable? Well, Wikipedia isn't so sure.
