I think the genuinely correct take here is that three things are true at once: 1. Absolutely you need confidence in any technology you use in high-stakes situations 2. Whether they come from AI or any other source, opaque certifications do not lead to trust 3. But you can build trust in complex systems through transparency and layers A good analogy to what is happening in mathematics now (and no doubt soon in biology and therapeutics) is what has happened over a couple of decades in open source software. The Linux kernel is extremely complex. In the beginning, it was not reliable. And it has led to numerous bugs and security risks. Even now, very few engineers understand all of it. Yet today, most of the critical infrastructure we use in the cloud is based on Linux. We have developed a transparent system for building layered systems of trust, managing risks by having different sets of people understand different components. Even if model capabilities are exponential, the actual development of these layers will take time, just like they always do. We will not instantly get magic solutions via AGI. If we treat any technology like a magic black box, we will have failures and accidents. But if we roll it out in a sane way, managing risks and understanding what we're building, we all benefit, just like we have with dozens of other extremely complex technologies, from software to gene therapy.