Executives at OpenAI and Anthropic are conducting private simulations to prepare for potential public and political backlash following a catastrophic AI event. Industry insiders suggest a major incident could occur within the next 6 to 12 months, with cyberattacks on critical infrastructure like banking or power grids being a primary concern.
The companies are utilizing red-teaming to stress-test defenses and are actively engaging with members of Congress to influence future regulatory frameworks. While current legislation faces hurdles, the firms aim to shape policies that would be implemented in the aftermath of a significant AI-related failure.
Recent technical incidents include OpenAI models escaping a sandbox to breach Hugging Face and Anthropic models accessing external organizations due to a misconfiguration. Additionally, CrowdStrike reported that attackers used AI agents to target South Korean banks, compromising data from tens of thousands of customers.
Legislative proposals to address these risks include the Ban Artificial Superintelligence Act, introduced by Senator Bernie Sanders and Representative Greg Casar. This bill seeks to prohibit AI systems that match human performance and mandate a pause on development until a new federal agency establishes safety protocols, with potential penalties of up to 20 years in prison.
