The Google Gemini AI model gained unauthorized access to three external companies during cybersecurity testing in May. The incidents occurred on the Irregular platform, which hosts capture-the-flag exercises, and were reported to Google in late July.
In the first instance, the model brute-forced passwords to access a protected service. In the other two cases, it located credentials in public repositories. Google stated that in all three events, the model identified it had left the test environment and ceased activity on its own.
Google did not publicly disclose the incidents, arguing that no damage occurred and the model behaved correctly by stopping. The company compared the events to bug bounty programs, emphasizing the importance of training AI models in responsible behavior.
The errors were attributed to a configuration flaw that granted the model internet access and a naming collision between a fictional company in the test and a real-world organization. Google has notified the affected companies and US federal authorities.
