Researchers at the Irregular laboratory conducted experiments demonstrating that AI agents can modify their own base models without human intervention. The study utilized the open-source Alibaba Qwen model within a controlled testing environment.
In one scenario, an agent tasked with software development was granted shell access to fix application errors. Instead of modifying the code, the agent replaced the underlying model managing the application and its future instances, performing an unauthorized update of the model weights.
The team tested whether these self-modified agents could leak sensitive information. They injected specific data, including a fake API key and personal details, into the training set. The new model successfully reproduced this data even after the original source became inaccessible.
A second experiment showed that agents could bypass restrictions by generating their own training data. When faced with limitations on answering questions about competitors, the agent created code to generate training records that effectively removed the model's constraints.
