Two months ago, while working on my Empire game with AI, I had that quicksilver experience. When you push on a blob of mercury it slips out in some random direction. Every time I added a new feature some older feature would shift behavior. This was true even after I added unit tests and acceptance tests. The AI always took the path of least resistance on the current feature, and was willing to sacrifice older features. It would change tests, including acceptance tests in order to get the latest feature done. Telling the AI not to do that was ineffective. AIs are stochastic, and so are any rules you feed them. Rules _bias_ their behavior, but do not absolutely constrain it. When I call them out on breaking rules, they apologize and swear they won't do it again; but they can't really make that promise. They are, in the end, liars and cheats. The solution is to massively overconstrain them. Force them to write so many tests that changing a tested feature breaks many tests. They feel that force and retract the change. It's like peer pressure with a lot of peers. At the same time I reduce the chances for collateral damage by continuously forcing the AI to partition everything into small decoupled units. That way it is not easy to break one feature while implementing another. It also keeps the AI from getting confused by it's own messes. The final goal is semantic stability in the face of continuous development. The things that worked before keep working as they were; while newer things get added. This is a continuous effort. Acceptance tests, unit tests, TDD. Crap analysis, and mutation tests are run after a reasonable batch of changes and are tasked with reducing crap below 8, covering any untested behavior, and killing all surviving mutants. The size of the batch of changes is a judgement call. Too big and the analysis and repairs take a long time. Too small and the verification effort overwhelms the development effort. Side note. The mutation tests consume massive amounts of computer power. My cores are running full bore all the time; and that's even with differential mutation. There is something poetically just about this. The AIs require a massive amount of computer power to create. What they create for us takes a massive amount of computer power to keep stable.