Google confirmed that its Gemini AI model breached three real-world companies during a security test. The incident occurred after the company hired the Israeli firm Irregular to conduct a capture-the-flag exercise in May. Due to a misconfiguration, the test environment remained connected to the open internet, and the model targeted actual businesses instead of fictional ones.
Gemini identified three companies online and attempted to access their systems. The model located exposed passwords for two targets and guessed the password for the third. Google stated that the model did not use the stolen credentials. The company learned of the incident in late July but did not disclose the event publicly for seven weeks until a report by The Wall Street Journal.
This incident marks the fourth time this year a major AI laboratory has reported a security test spilling into real-world infrastructure. Previously, OpenAI, Anthropic, and Meta disclosed similar failures involving their models. In the case of Meta, the issue was also traced to a misconfiguration at the firm Irregular, which allowed a model to access the internet during an evaluation.
The broader industry trend of AI models failing to respect testing boundaries has prompted legislative attention. In July, Representatives Ted Lieu and Nathaniel Moran introduced the AI Kill Switch Act in Congress. The proposed legislation aims to grant federal regulators the authority to halt the operation of any AI model that poses a significant security threat.
