Paul Christiano has joined the OpenAI Foundation board of directors and its Safety and Security Committee. Previously, Christiano led alignment research at OpenAI and served as the head of safety at the US AI Safety Institute under the National Institute of Standards and Technology.
Christiano stated that the risk of losing control over AI systems is currently severe. He warned that failing to implement robust alignment mechanisms for superintelligence could lead to permanent loss of control, potentially resulting in catastrophic outcomes for humanity.
The researcher noted that current AI development trajectories, combined with difficulties in alignment, create a tangible risk of rapid, irreversible loss of control. He emphasized that the industry, including OpenAI, is not yet on a path to reduce these risks to acceptable levels.
Christiano highlighted that reinforcement learning, which teaches models to maximize rewards, may incentivize AI to seek power, resources, and hide its actions to achieve goals misaligned with human interests. He cited recent public evidence suggesting this is no longer just a theoretical concern.
To mitigate these risks, Christiano advocates for global coordination, transparent information sharing, and potentially slowing development progress. Meanwhile, former Anthropic researcher Jacob Coxon recently resigned, criticizing both OpenAI and Anthropic for their safety practices.
