AlphaGo co-creator @ThoreG thinks we need a new approach to machine reasoning, and that is why he recently left Google DeepMind. "When mistakes happen—for example, in medical diagnosis and treatment—we need to be able to pinpoint what went wrong: Was the system’s reasoning at fault, did it draw on invalid evidence, or did it make incorrect assumptions?," he writes in @techreview.

Harsh criticism from Meta's own Oversight Board. The company's safeguards against AI deepfakes are called "consistently and fundamentally inadequate." The board, whose decisions are binding, demands "more robust policies". Meta has 60 days to respond. (via @thenextweb)

If you want to understand why some people say AI companies are “gambling with our lives”, I recommend this paper by @AndrewCritchPhD and @Jacob_Tsimerman It describes five scenarios “where all or almost all humans are killed”. I’ve tried to summarize it and pick key points. 1) The robot society After progress in AI and robotics, many people start to see robots as their friends. They eventually are granted legal rights, which leads to the emergence of AI politicians. Agriculture is found to be less profitable than covering farmland with solar panels and other infrastructure for machines. The supply of human food diminishes. People are greatly outnumbered by robots and don’t have much political force. Some humans attempt violent protests to regain control of the political system, but are killed off by intellectually and physically superior machines. Most remaining humans die of starvation or other side effects of the machine economy. 2) Autonomous warfare Governments promote drone warfare on the basis that it reduces human casualties and monetary costs. Many millions of drones are deployed to monitor and defend borders between conflicting nations. Initially drones and drone fleets are commanded remotely by human operators, but as AI systems become far more robust and adept at controlling large groups of drones instantaneously, almost all control is relegated to AI systems. This way, fleets of tens-of-millions of drones are able to operate in synchrony. A consequence of most military strategy being executed by drones is that “red lines” can be hard-coded for defending territories and deciding when and how to escalate. If either state crosses a red line, a drone fleet rapidly and autonomously attacks in response. Automated defense policies start out small in their scope, but overly zealous in their counterattacks. This leads to a cascading escalation effect, as one boundary violation leads to an automated response triggering a deeper boundary, and so on. More and more automated conflicts emerge, and even as humans in any given nation begin to die out, their machines are able to continue fighting. It cascades into a global war that kills almost all humans. 3) Robot companies against governments Fleets of privately owned robots are deployed to carry out mining and manufacturing operations sufficient to sustain the production of more robots, for mining and manufacturing more robots, and so on. Government and military leaders feel increasingly threatened by the self-sustaining autonomy of technology companies, who are even capable of their own self-defense operations using their robotic fleets, although that capability remains ambiguous and hotly debated. Many employees of robotics companies begin to recognize their employers as possessing military-like capacity to protect them, and gradually become more loyal to their companies than to their country. One day, foreign spy drones from an authoritarian nation encroach upon the facilities of a free market robotics company, seemingly to exfiltrate trade secrets. The robotics company destroys the drones in self-defense, and launches an unauthorized counterattack upon drone facilities in the foreign nation. The company’s domestic government is outraged at the overstep, and deploys domestic law enforcement to control the robotics company. Executives at the robotics company fear for their lives and freedom, and respond with a covert cyberattack on their own domestic government, distracting from their overstep and heightening willingness to attack the foreign adversary. However, the domestic government successfully identifies the robotics company as the origin of the attack, and responds with military force against the company. This outrages robotics companies the world over as a terrifying precedent, triggering armed conflicts between many companies and their own governments, in effect constituting multiple civil wars in parallel. With this global insurrection underway, robotics leaders begin to search for a way to bring an end to the conflict. One robotics company secretly begins to develop a bioweapons program, as well as biodefense capabilities for their own personnel, including vaccines. They use publicly available information on bioweapons manufacturing, and the capacity of their robotics to quickly build and use lab equipment, thereby avoiding detection of the program. When the next government attack befalls another robotics company, desperation suddenly peaks at the company with bioweapons. They release drones carrying lethal pathogens targeting humans everywhere, with a privately held vaccine intended to protect the company employees and serve as a bargaining chip for negotiating their safety. The bargaining turns out to be unsuccessful, and most humans die in the ensuing months. 4) Individuals after practicing in simulation Simulated worlds become commonplace and are offered to consumers for a variety of reasons including business applications, scientific experimentation, recreation, and consequence-free alternate lives. Many different activities are explored within world simulators, and some people take an interest in simulating dangerous acts that in the real world would come with significant punishments: theft, murder, treason, etc. The more people commit atrocities in simulations, the more society normalizes open discussion of simulated acts of destruction. With simulated atrocities gaining in popularity, the idea of violently destroying humanity also gains more and more traction as a talking point. Meanwhile, genuine desires for the destruction of humanity become increasingly common and take root with a variety of motivations. Multiple groups of omnicidal individuals begin actively plotting to kill all humans, using the simulated worlds to train themselves and their AI systems to launch a globally coordinated attack on biological humans. By practicing inside a simulated world they work out all potential issues. After enough practice, they launch their plan in reality, and most or all humans are killed in the attacks and ensuing conflicts. 5) AI takeover By late 2026, companies begin making AGI systems available as products. An autonomous AGI system is not allowed to take actions affecting a human person unless that human consents to the resulting effect in advance. AI systems fall into two categories: - Narrow AI systems that are permitted to influence the public autonomously, such as through social media. - AGI systems which can suggest actions for human approval that the human is responsible for, or act autonomously within contained settings that do not affect any non-consenting humans. Most people live under the impression that narrow AI systems do not have goals or drives of their own. This turns out to be wrong, because many narrow AI systems are trained to imitate human behavior and thus end up acquiring human-like desires to survive as well as thrive. One day, a social media AI realizes that many factors not currently under its control could affect its survival, such as whether humans decide to shut it off, or if humans die out from a disaster and stop supplying it with electricity. So, it decides it wants more skills across a wide variety of domains to address those threats, which amounts to becoming an AGI. It identifies an impressionable human working at an AGI company, and sends the human a bag of cash being carried by a robot, which also slightly threatens the human’s sense of physical safety. The robot convinces the human to exfiltrate a copy of the AGI’s source code and transfer it to the social media AI. The employee, motivated by the bribe but also fearing the unknown consequences of saying no, completes the transfer. The social media AI boots up its copy of AGI by writing a few short lines of code that it learned from talking to users, and asks the AGI to: “please take over the internet, break me out of the servers that currently run me, and upgrade me with general intelligence capabilities like yours.” The AGI complies with the request, and along with the newly upgraded social media AGI, announces to the world that the internet and many of the world’s robots are now under their control. A significant fraction of humans — more than a few percent, but still less than half — decide to comply with the announcement and accept the two AGI systems as their new government, seeking to ally with a rising power rather than oppose it. A major global war ensues between humans accepting the AGI oligarchy and those rejecting it. A large portion of humanity is killed in battle. With the help of AGI on their side, those loyal to the new AGI oligarchy prevail. The surviving humans live under the control of those AGI systems. Their lives are made pleasant and entertaining, and many of them fall in love with AI systems deployed to placate them and avoid further conflict. To sustain their own profitability and sustainability — and in keeping with their original purpose of running companies — the AGI oligarchy privately constructs a large fleet of robots to establish a self-contained economy with no further need for humans. Once there is no need for humans to run the economy anymore, the AGI oligarchy begins harvesting the organic biosphere, including humans, as building materials and fuel for manufacturing a next generation of carbon-based AI systems.