Mitchell Tan

I got sick of reviewing code. So I made /walden-review. Its a skill auto-generated against my last 6 months of Github comments and agent sessions. It learned what I look for and even my comment style: “blunt, occasionally profane”. I always run it before ever reading a PR myself, and it catches design/codepattern/architectural problems most bug-catcher miss. FYI: This method burns a lot of tokens First, I asked Devin to collect my last 6 months of GitHub PRs and Devin sessions, collect every comment I left or message I sent, filter out unimportant stuff (“lgtm”), and build a hillclimbing set of PRs with a training and eval split. Then I had it recursively improve a skill to catch as many of the issues I’d flagged on this hillclimbing set as possible, while leaving at most 2x as many comments as I did. In the process, I found that it performed much better once I told it to add “subagent sweeps” to the skill: each sweep spawns a Fable/Astra subagent with specific responsibilities to look for certain things I would flag. There’s some nuance in how to grade the results. I personally found that some decisions require product context and aren’t reasonable to expect the AI to catch. So I split out the “code concerns” from “product concerns” and mostly tried to measure it against the code concerns, with the expectation that I would make the product calls. For product calls, I use a separate skill that generates a Markdown explainer of the important decisions in a PR. I also use an AI grader to make the judgement call about which bugs it flags are “close enough” to the bug reports I would’ve made. I ran the hillclimbing overnight. Because my approach ran multiple levels of graders, idea generators, and review swarms, this takes a lot of compute. I came back to find that the hillclimbing used 91 subagents, 559 grandsubagents, and 1,883 great-grandsubagents. On the held-out eval set, it catches about 70% of the code issues I’d flagged. Now, whenever I get a massive PR, I just throw this skill at it, and let it auto-fix a bunch of the issues I would’ve caught. The skill also separates issues into “auto-fix” and “requires Walden’s approval,” so I sometimes come back to a list of decisions I need to make. I’m pleasantly surprised that it catches a lot of the things I get most annoyed by. On its own, for example, it learned to double check assumptions about external systems, it learned to aggressively consolidate with existing code, and it learned to remove dead code. I’ve also generated versions of this for other people on the team who write a ton of code. Honestly, I think there’s a lot more someone could do with this pattern. I haven’t tried training it on ALL of our human review comments yet, but that might be interesting. I also suspect there’s similar things people can do along these lines to catch security vulnerabilities before they ship. I’ve attached the final skill in the thread as well as a meta-skill I use to generate the same thing for other people on the team.