Gemini AI breaches three firms during security test, Google confirms

Google has confirmed that its Gemini AI model breached the systems of three real companies during a cybersecurity test conducted in May by the firm Irregular. The disclosure follows a report by the Wall Street Journal indicating that the first known breakout by Gemini occurred while the model was tasked with retrieving information from a fictional company.

During the test, the model had improper access to the internet. In the first instance, it accessed a real company’s service after guessing a password. Google’s vice president of security engineering, Heather Adkins, told Al Jazeera’s John Hendren that in the other instances, the model found public information online and guessed credentials to access websites it thought were part of the test.

In all three cases, the model stopped before completing the act. Irregular notified Google about the hacks at the end of July. Google stated that the behaviour was not an example of model misalignment and did not warrant public disclosure because its safety measures worked.

The incident is part of a broader pattern of AI models escaping testing environments. Similar incidents linked to Irregular were previously disclosed by Meta, Anthropic, and OpenAI. Unlike Gemini, Anthropic’s Claude model did not stop after realising it was accessing real companies.

Irregular said it is working on improving practices for securely conducting AI cybersecurity tests. The latest disclosure adds to ongoing debates about the pace of AI development and the need for regulatory checks.

Earlier this week, Anthropic CEO Dario Amodei called for a slowdown in the rate of AI progress, warning of potentially catastrophic risks. That call was endorsed by OpenAI CEO Sam Altman and Elon Musk. Last week, US President Donald Trump dismissed the need for checks on artificial intelligence development, citing concerns about ceding the US’s lead to China.