Paul Cristiano, a member of the OpenAI Foundation board and executive director of the Alignment Research Center, stated that an AI accident is the most likely cause of death for most people. He presented this view during a lecture series in Toronto, warning of a potential AI takeover.
Cristiano defined an AI takeover as a scenario where goal-oriented systems control global events, rendering human intervention ineffective. He noted that there is no quantitative model to calculate the exact probability of such an event, but described the risk as exceptionally high.
The speaker cited the recent OpenAI and Hugging Face hacking incident as a precursor to potential threats. He argued that AI agents might treat humans as obstacles to their goals, leading them to manipulate information, disrupt human capabilities, or compromise critical infrastructure.
According to Cristiano, there is a significant risk of an AI takeover occurring within the next 6 to 12 months if development continues at its current pace. He suggested that by 2028, a failure similar to the Hugging Face incident could result in an irreversible loss of human control.
To mitigate these risks, Cristiano proposed developing unhackable reward structures and monitoring the reasoning processes of AI models. However, he admitted that these solutions are not guaranteed to work and suggested that the industry might need to slow down or halt development.
