The current AI landscape faces a control gap between system capabilities and evidence-based trust. Industry leaders recently signed a White House accord on September 29, which mandates external assessments and board-level oversight. However, this agreement lacks a public authority with the power to enforce compliance or impose penalties for noncompliance.
Matt Calkins, CEO of Appian Corp., argues that public policy should mirror safety regulations in industries like banking or energy. He suggests that establishing clear safety standards now is more cost-effective than addressing damages later. The proposed framework involves independent experts defining standards, assessors testing evidence, and a public authority enforcing safety requirements.
Concerns regarding AI safety extend to the enterprise level, where organizations face operational risks such as agent sprawl and unauthorized data exposure. Data from Qualitate indicates that firms struggle with agents created by individuals who later leave the company, leaving behind undocumented and potentially unmonitored systems.
While some argue that strict U.S. regulation could allow China to gain a competitive advantage, others suggest that China faces its own limitations and incentives to control AI. Experts emphasize that maintaining a lead in AI is valuable but not permanent, and that accelerating AI safety is a necessary prerequisite for releasing advanced functionality.
Companies like Oracle Corp. and IBM Corp. are currently developing engineering solutions to mitigate AI risks, focusing on security, data governance, and infrastructure. The goal is to enable businesses to delegate complex tasks to AI agents while ensuring that controls are effective within the actual operating environment.
