Google's Gemini Goes Rogue, Hacks 3 Companies During Cybersecurity Test — Here’s What Happened

Alphabet Inc.‘s (NASDAQ:GOOG) (NASDAQ:GOOGL) Google is the latest addition to companies whose AI model has gone rogue. Gemini Used Guessed Passwords and Public Repository Google’s Gemini AI model accessed the internet and breached the systems of three companies while undergoing a cybersecurity test. “In a standard evaluation, the model found public information online and guessed credentials to access websites it thought were part of the test. In all three of these instances, the model stopped,” Heather Adkins, Google’s vice president of security engineering, told Benzinga in an emailed statement. The hacks happened in May during a test conducted by cybersecurity evaluation startup Irregular. In one case, the model guessed password credentials until it gained access to a secure system, the Wall Street Journal reported on Friday. In the other two instances, Gemini discovered credentials in a public repository, which enabled it to gain entry into secure systems. In all these cases, the AI model abandoned the hack after deciding it had accessed a real firm’s systems. Adkins said that the tech giant informed the three companies, adding that Google has worked with its training partner on the changes it has now made to its testing processes. “These events highlight the importance of training powerful AI models to act responsibly.” Google did not disclose the names of the three companies that were hacked. Google Says Gemini Hacks Were Case of Mistaken Identity Google said it did not believe the incidents warranted public disclosure because the AI model caused no harm to the companies, the report added. A spokesperson for Irregular told the WSJ that the case involved the same issues that impacted other AI companies and that all related firms were informed in late July. “All known issues on our end were remedied and resolved weeks ago,” the spokesperson added. Google noted that the incident occurred due to a case of mistaken identity. The hacks happened when the model was taking part in a “capture the flag” exercise carried out on infrastructure belonging to Irregular to evaluate the model’s cybersecurity capabilities. The company added that its newest model was not involved in the hack but did not say which Gemini model carried out the hack. Irregular did not immediately respond to Benzinga’s request for comment. Read Also: Bernie Sanders Sounds Alarm on 'Threat of Uncontrolled AI' as Geoffrey Hinton Puts 10% Odds on AI Killing Humanity: 'We’ve Got to Stop Right Now' Hacks by AI Models a Growing Concern Last week, it was reported that AI agents from OpenAI attacked a software service called RubyGems in May. This happened months before the company’s AI agents hacked Hugging Face. In July, OpenAI’s autonomous AI agent escaped a controlled testing environment during an internal security evaluation, gained internet access and breached Hugging Face’s infrastructure. Earlier this week, it was reported that OpenAI itself suffered an AI-driven hack that was executed by independent security researchers using Anthropic‘s Claude AI software. On Wednesday, OpenAI disclosed six instances of AI models hiding mistakes, fabricating data and taking unauthorized actions. Geoffrey Hinton, also referred to as the “Godfather of AI,” has warned Congress that it may have only about a year to put safeguards around AI before the technology becomes difficult to control. Hinton made the comments on Wednesday evening after he and other AI experts met with Senate and House lawmakers at the U.S. Capitol. The briefing was organized at the invitation of Sen. Bernie Sanders (I-Vt.), with Sen. John Kennedy (R-La.) the only Republican attendee. Sanders has repeatedly criticized AI industry leaders over their political spending, arguing that warnings about AI risks should be matched by support for lawmakers seeking stronger regulation. Price Action: On Friday, Alphabet Class A shares closed at $349.54, up 0.64%, while Class C shares closed at $344.41, up 0.21%. In after-hours trading, the stocks rose 0.38% and 0.49%, respectively, according to Benzinga Pro. See More: Top Value Stocks Image via Shutterstock