Actually, Tao is right and this is being misread. He’s talking about keeping human understanding of how the AI got there, so we can detect misalignment (e.g. gaming a weakness to mask a result that isn’t truly a good cure). Verification of the proxy alone doesn’t guarantee the real outcome is safe or aligned. The AI could be gaming the system to satisfy the request, or worse. Without understanding how it arrives at its results, we lose the ability to spot and correct such failures before they scale. This differs from not fully understanding how something like Tylenol works, because the intent and process behind those were human and aligned with our goals.