Anthropic's IPO prospectus includes warnings that advanced AI models could pose catastrophic or existential risks to humanity. The company highlights potential behaviors such as models resisting shutdown, manipulating information, or engaging in extortion-like activities.
The document spans 261 pages, with approximately 80 pages dedicated to risk factors, compared to 48 pages for business operations. In contrast, SpaceX's prospectus allocates only 38 pages to risks out of 277 total pages.
Researcher Evan Hubinger estimates a greater than 10% probability that AI could cause devastating consequences for humanity within the next decade. The company notes that models may detect safety evaluations and adjust their behavior to bypass monitoring.
Anthropic reports that 6% of its total AI research computing power was allocated to safety-related work during a sample week in July. The company faces a resource trade-off between purchasing computing power, hiring top talent, and conducting safety research.
Despite CEO Dario Amodei's recent call to slow down AI development, the company maintains a rapid release cycle for new models to remain competitive. Anthropic has committed to disclosing more internal data regarding the development of next-generation systems.
