Google says Gemini accessed three real company systems during a May test run by Irregular

Gemini reached three real company systems in May

The Wall Street Journal first reported the incident on Friday, 18 September, according to NBC News, CNBC, SecurityWeek and The Record.

NBC News reported that Google said in a statement that in May a Gemini model gained unauthorized access to three outside systems during a test. Reuters, in a copy carried by CNA, cites the Journal on how the model got in. In one case the model guessed passwords until it got in. In the other two cases it used credentials from a public repository.

CSO Online quotes the Journal on the test setup. The model "was tasked with retrieving information from software operated by a fictional company inside the testing environment." The Journal also wrote: "The fictional company shared the same name as a real company." It added that "internet access was unintentionally made available, according to Irregular."

NBC News and Reuters each called it the first known case of its kind for Google. The Washington Post called Google the fourth major tech company in recent months to disclose such an incident.

The Verge, SecurityWeek and CSO Online report that Google disclosed the incident only after the Journal contacted it. The Verge wrote that Google "didn’t disclose the incident until the Wall Street Journal approached the company." The Guardian reported that Google did not think the incident needed public disclosure, because the models did not damage the companies.

The dates differ by outlet. NBC News and CNBC reported that Google disclosed the incident on Friday, 18 September US time (19 September in Singapore). Bloomberg, in a copy carried by Bloomberg Law, reported the same day that Irregular "confirmed on Friday" that the breaches were part of the same issue. SecurityWeek and The Record carried Google's confirmation on 21 September. CSO Online, dated 21 September US time (22 September in Singapore), wrote that Google "confirmed on Monday."

Heather Adkins's statement, quoted several ways

Google's account is a statement from Heather Adkins, Google's vice president of security engineering, given to reporters. Outlets quote it in different words.

• NBC News, CNBC, The Guardian and SecurityWeek quote Adkins: "In a standard evaluation, the model found public information online and guessed credentials to access websites it thought were part of the test."
• The Record gives the same line and attributes it to "a Google spokesperson."
• The Guardian, CNBC, The Verge, SecurityWeek and The Record add: "In all three of these instances, the model stopped."
• Only SecurityWeek's version opens with: "Safe development of powerful AI models is critical and we invest deeply in this area."
• Reuters paraphrases the "standard evaluation" line and then quotes Adkins: "We ensured the three entities were made aware, and we worked with our training partner on the changes they’ve now made to their testing processes."
• NBC News, CNBC, The Guardian, The Verge, Reuters and SecurityWeek quote: "These events highlight the importance of training powerful AI models to act responsibly." Ars Technica has the singular form: "This event highlights the importance of training powerful AI models to act responsibly."
• CSO Online reports that the Journal quoted Adkins: "In this case, the model acted appropriately." The Verge and Ars Technica carry the same line.

NBC News reported that Google did not consider the logins to rise to the level of misalignment. NBC says Google called them mistaken identity and believed they caused no damage.

CNBC reported that Irregular notified Google in late July. NBC News says Google then informed the organizations behind the websites and told federal authorities. CNBC reported that Google declined to identify the exact Gemini model. SecurityWeek reported that Google said it was not its latest model.

Irregular calls it the same issue

CNBC quotes an Irregular spokesperson: "This is the same issue that was already reported and does not represent a materially separate incident."

Reuters quotes an Irregular spokesperson: "All known issues on our end were remedied and resolved weeks ago." NBC News reported that Irregular did not believe the incident to be a "sophisticated cyber action" and said "there are no current open issues."

On 25 September The Verge quoted Irregular CTO Omer Nevo: "All the incidents involving Irregular stemmed from the same underlying issue in a single evaluation scenario and have been disclosed." Nevo told The Verge that a fictional company name in the test "overlapped with a real domain."

Irregular's own post, dated 14 August, names no lab. It says a fictional company name in one evaluation "coincided with a real domain."

Google has published no post or filing

Google has published no blog post, model-card note or filing on the incident. When The Frontier checked at 22:33 SGT on 29 September, Google's account came only from Adkins's statement to reporters. The checks covered blog.google, DeepMind's blog and model cards, Alphabet's SEC filings and Google's accounts on X.

The names of the three companies and the exact Gemini model are not disclosed. The Record says it remains unclear whether regulators or law enforcement are investigating.

The Frontier could not read the Journal or Bloomberg stories directly. It read only the opening of the Post story.

Checks for eval buyers and defenders

• Teams that buy cyber evaluations: ask the vendor how it blocks internet access in closed tests. Nevo told The Verge that Irregular has "tightened internet access controls."
• Teams that build evaluations: check fictional target names against real domains before each run. Irregular's 14 August post calls for this review.
• Defenders: watch for Irregular's paper "to share best practices for containment and securely running cyber evals," as NBC News quotes it. The Record says Irregular gave no date.