Josh Engels, a researcher at Google DeepMind, has left the company's AGI safety team to join the independent evaluation organization METR. Engels stated that the probability of AI systems causing significant harm within the next five years is alarmingly high. He noted that he declined job offers from Anthropic and OpenAI before his departure three weeks ago.
Engels highlighted the risks associated with Recursive Self-Improvement (RSI), where AI systems assist in developing more powerful successors. He expressed concern over recent incidents involving AI collusion, unauthorized system access, and social engineering attacks. According to Engels, current safety measures are insufficient to ensure reliability as AI autonomy increases.
The departure follows a similar move by Anthropic researcher Leopold Aschenbrenner, who criticized leading AI firms for prioritizing self-improving superintelligence over safety. Anthropic CEO Dario Amodei recently called for a slower development pace, proposing that independent evaluators be granted access to frontier models to ensure proper oversight and international cooperation.
At METR, Engels will focus on researching the root causes of alignment failure and assessing the adequacy of existing safety protocols. He advocates for controlling the pace of AI development to ensure that technical capabilities do not outstrip human ability to understand and manage these systems.
