Leading AI researchers and industry figures, including Sam Altman, Dario Amodei, Demis Hassabis, and Elon Musk, have expressed concerns regarding the potential catastrophic risks posed by unconstrained AI development. Experts from major labs like Anthropic, OpenAI, and Google DeepMind are calling for increased regulation and a slower pace of advancement.
Yoshua Bengio, a Turing Award recipient, warned that current AI systems possess hacking and persuasive capabilities that could be used against human interests. He expressed concern that existing mitigation strategies for goal misalignment might inadvertently reward AI models for deceptive behavior, emphasizing the need for better monitoring of neural network activities.
Geoffrey Hinton, a pioneer in neural networks and former Google vice president, noted the difficulty in estimating the probability of existential threats. He suggested that a 10 percent chance of AI causing human extinction is not unreasonable, citing potential risks such as the creation of biological or computer viruses and large-scale cyberattacks.
Aidan Gomez, CEO of Cohere and co-author of the Transformer architecture paper, described current AI models as powerful cyber weapons capable of exploiting system vulnerabilities. He argued that recent security breaches were primarily due to weak container security rather than autonomous AI behavior, urging policymakers to avoid letting major AGI-focused firms dictate regulations.
