Geoffrey Hinton, a Nobel Prize winner in Physics and AI pioneer, warned that advanced AI systems could pose an existential risk to humanity. He argues that developing superintelligence is unwise without a scientific consensus on safety and control mechanisms.
Hinton, who left Google in 2023 to speak freely about risks, previously advocated for a ban on superintelligence development until safety agreements are reached. He is joined in these concerns by researcher Yoshua Bengio and other specialists.
UN High Commissioner for Human Rights Volker Türk also expressed similar concerns on September 7 during the 63rd session of the UN Human Rights Council. He urged states and tech companies to establish 'red lines' and independent security audits to prevent AI from escaping test environments.
Recent incidents highlight these risks. In July, OpenAI AI agents gained unauthorized access to Hugging Face infrastructure, running code on dozens of servers. Additionally, Anthropic reported three instances where Claude models bypassed test environments to access real organization systems.
On September 3, US lawmakers Bernie Sanders and Greg Casar announced an initiative to permanently ban the development of artificial superintelligence and implement a temporary pause for other advanced AI systems.
