Speaking of metastable failures, one of the longest open tabs in my browser is this. Yet to read the post, but wonder how much of this methodology overlaps with what you did for Aurora. Only way to find out is to get around to reading the literature. arxiv.org/pdf/2606.00942

I found this unpublished blog post from 6y ago (look at the hand drawn diagram lol). I’d totally forgotten about this, but the key ideas here (embracing non-determinism as a first class citizen via “error kernel” centric design etc) work super well as guidelines for my agents.

Lot of it holds up pretty well, actually. The main changes in the last ~7 years have been in: - move towards more flexible consistency - use of replicated logs for all manner of things - move away from strict consensus and synchronous replication - control plane design choices

I started this series on testing nearly a decade ago, but never published the final part copyconstruct.medium.com/tes… There’ve been so many great advances in the storage space in the ~7y I feel this is worth completing. Have a draft from years ago I hope I can wrap up in early 2027.

How to effectively debug with agents when you no longer read or understand the code In this post, I propose a debugging framework for agents that doesn’t rely on human knowledge of code to get to the right “root cause” The TL;DR are the screenshots. Link to post in the reply.

I’ve found that agents are great at debugging any problem when provided with this two-tier framework for troubleshooting. This requires enforcing discipline and imposing constraints, rather than give agents a free rein or vague end goals, but works better than unguided “loops”.

Ball-knowers knew all along that Twitter wasn’t just going to collapse after Elon fired 75% of the staff, because Twitter 1.0 had some of the best infrastructure engineers working on it, and when you get that calibre of people building your foundations, it lasts.

With agentic coding, even in the best case scenarios (where you read every line of generated code), there’s something we’ve lost, perhaps irrevocably. That has a real cost, & not to mention, second and third order effects, that “software factories” etc don’t necessarily solve.

Writing a post about “how to effectively debug code when you no longer fully understand it”. This is real bug I had to fix in ~9K LOC (100% AI generated). None of the tactics I started with here helped “root cause” the issue, lol. Had to take a different approach to isolate it.

One thing I don’t see many talk about is how coding agents can often *amplify* weaknesses, both individual and organizational. I think of this as the “empty calories” problem, which is exacerbated when “hyperproductivity” originates from your team’s “lowest common denominator”.