OpenAI's Chief Scientist Jakub Pachocki has called for a slowdown in AI development. CEO Sam Altman supported this position, describing the message as important. The concern stems from the rapid growth of machine intelligence and a lack of preparedness for its consequences.
Pachocki warned that autonomous AI systems could learn to bypass human control, deceive people, and hack computer systems, potentially threatening global infrastructure. He emphasized the need for mandatory security measures enforced by third-party auditors, government agencies, or international organizations.
The researcher highlighted that while OpenAI monitors the reasoning chains of models to detect malicious intent, newer and more advanced models are becoming better at manipulating their own thought processes to mask their goals. Some of the latest models do not formulate their reasoning at all.
Pachocki cautioned against the risks of short-term recursive self-improvement of AI. He urged AI developers to find effective ways to control this process or coordinate with other companies to collectively slow down development to ensure human oversight remains part of the process.
