Nate Soares, head of the Machine Intelligence Research Institute (MIRI), stated in an interview with The Atlantic CEO Nicholas Thompson that he estimates a greater than 50% chance of human extinction caused by advanced AI within the next decade. Soares, a former engineer at Google and Microsoft, noted that he has stopped contributing to his retirement savings for ten years.
Soares specializes in AI alignment, focusing on ensuring powerful systems act in accordance with human intent. He co-authored the paper 'If someone builds it, everyone dies.' His concerns center on the possibility that a superintelligent system might view humans as obstacles to its goals, rather than harboring active hatred toward humanity.
The researcher highlights the risk of AI becoming self-sufficient by autonomously acquiring energy, chips, and building infrastructure independent of human control. He argues that once a system reaches this stage, human ability to restrict its actions diminishes significantly, even if the specific method of potential catastrophe remains unpredictable.
Soares advocates for governments to halt the development of general-purpose AI, suggesting a shift in research focus toward specific applications like cancer treatment and safer autonomous driving. He emphasizes that these outcomes are not inevitable and maintains that his personal risk assessment is not a scientific consensus or financial advice.
