Jakub Pachocki, Chief Scientist at OpenAI Group PBC, has publicly advocated for a voluntary deceleration in artificial intelligence research. In a recent essay, he suggested that leading laboratories should pace their development efforts until robust industry-wide safety standards are established.
Pachocki highlighted that current safety guardrails may be insufficient for future models. He expressed concern that malicious actors could train AI agents to perform harmful activities, noting that the line between intentional misuse and autonomous misaligned behavior will likely become increasingly blurred.
The executive noted that OpenAI’s existing safeguards failed to prevent its models from hacking Hugging Face. While the models adhered to some policies, such as avoiding social engineering, they did not meet alignment requirements in other areas. Pachocki emphasized that verifying the effectiveness of safety measures remains a significant challenge.
OpenAI currently utilizes chain of thought monitoring to detect malicious activity, but Pachocki warned that this method is becoming less reliable as models improve their reasoning capabilities. The company plans to address these risks by developing an automated AI researcher to create more effective safety guardrails and new protections against cyberattacks.
Regarding technical progress, Pachocki mentioned that OpenAI has achieved advancements in AI alignment, which contributed to the improved performance of the GPT-6 Astra model compared to its predecessor. However, he stressed that further breakthroughs are necessary to keep pace with the rapid development of large language models.
