For the first time, Google's Threat Intelligence Group caught a zero-day exploit built with AI assistance before attackers could deploy it at scale. The exploit targeted two-factor authentication on a popular open-source system administration tool, and Google said its proactive discovery "may have prevented" a mass exploitation event. The exploit came as a Python script designed to bypass 2FA on an unnamed web-based admin tool. Google worked with the affected vendor to patch the vulnerability, and the company's name, the tool, and the threat actor group remain undisclosed.
What tipped off researchers that AI created this exploit was not just the code itself, but how it looked. The script contained a hallucinated CVSS score (a fake severity rating that no human analyst would generate), excessive educational docstrings, and "textbook Pythonic formatting" that Google described as highly characteristic of LLM training data. The clean structure, detailed help menus, and an ANSI color class gave it away.
"The script contains an abundance of educational docstrings, including a hallucinated CVSS score, and uses a structured, textbook Pythonic format highly characteristic of LLMs training data," Google explained in its report.
Google does not believe its own Gemini models were used, but stated it has "high confidence" an AI model supported both the vulnerability discovery and the exploit weaponization. The exploit required valid user credentials to work. Attackers still needed stolen login details as a first step, then used the AI-generated bypass to circumvent 2FA protections. The vulnerability itself stemmed from a hard-coded trust assumption in the application's authentication system, a type of flaw that standard security scanners often miss.
John Hultquist, chief analyst at GTIG, told The New York Times the case represents "a taste of what's to come" and "the tip of the iceberg," calling it the first "tangible evidence" of AI-assisted zero-day attacks.
Google's broader report, drawing on data from Gemini, GTIG, and Mandiant, documented how state-sponsored groups are already integrating AI into offensive operations. The Chinese group UNC2814 used persona-driven jailbreaks (instructing AI to act as a senior security auditor) to enhance vulnerability research on embedded devices like TP-Link firmware.
North Korea's APT45 sent thousands of repetitive prompts to recursively analyze CVEs and validate proof-of-concept exploits.
"This results in a more strong arsenal of exploit capabilities that would be impractical to manage without AI assistance," Google said.
The report also covers autonomous malware operations, AI-improved defense evasion, and supply chain attacks. Google noted that while threat actors are using AI across vulnerability research, exploit testing, and malware development. The technology also works for defenders.
Last month, Anthropic launched Project Glasswing, using its Claude Mythos Preview model to find and defend against high-severity vulnerabilities.
