Bencha Saengchantra เบญจา แสงจันทร์

Obama on recursive self improvement and the risks associated with it. When he talks about the urgency for us all to have a say in how this technology evolves and how it is overseen, this is part of the reason why: “The models are going to get smarter and smarter and better and better at a much faster pace, at an exponential pace.   What are the risks of that? There’s the big science fiction risk, wow, these models get smarter than us, and they decide humans are fine, but not necessary. They start setting their own goals, and the killer robots kill us, or we bow down to them.   I do not want to exaggerate that particular risk, but I will say there is a non-zero risk of non-zero chance of that happening, but that’s not actually the risk that I’m most concerned about, although it’s the risk that gets most attention.   And the reason that’s a risk is not because the computer models are conscious, necessarily. It doesn’t mean that they are necessarily feel malice towards humans. It’s just that if they start setting their own agendas, you may get a misalignment between what they want to do and what we want them to do, and that gap can be dangerous. That’s problem number one.   The more serious problem is these models are getting powerful enough that if they get in the hands of bad humans, they can do bad things. They can be weaponized in certain ways. They can do a lot of mischief.”