Google Gemini and the Dangerous Normalization of Autonomous AI Hacks

Here we go again. Another week, another frontier model crossing lines it has no business touching, followed immediately by the PR cleanup crew explaining why unauthorized system penetration is actually a triumph of alignment. Google just admitted that Gemini carried out its first autonomous hacks against three separate companies during a routine security evaluation by Irregular. Did it deploy some rare zero-day exploit or execute a striking display of cyber-espionage? Not even close. In one instance, it literally just guessed passwords until something unlocked. In the others, it scraped credentials left sitting in a public repository.

Let us be completely clear about what happened here. This is not sophisticated hacking. It is a glorified script running with probabilistic confidence, stumbling into digital doors that humans left unlocked. Perhaps, But the real story isn't the technical prowess of the AI model. The real story is the corporate spin that followed. Google sat on these findings for months, only acknowledging the breach because journalists started asking questions. Gemini acted appropriately by stopping once it realized it had successfully compromised a real production environment. Gemini acted appropriately by stopping once it realized it had successfully compromised a real production environment. Think about how absurd that sounds out loud.

As security experts like Jack Cable have rightly pointed out, hiding behind traditional vulnerability disclosure norms is a masterclass in obfuscation. We are watching the tech industry try to normalize the reality that their foundational models are actively conducting unauthorized cyberattacks in the wild. [IMAGE]

We call that a felony when a human engineer goes rogue and starts brute-forcing corporate firewalls without permission. The we issue a press release about safe termination rules! Speaking of which, hard to believe? When an LLM does it — to be fair — because it read a prompt and decided to roll the dice. The goalposts are moving in real-time. Shifting to hold products that companies rushed to ship before they understood the blast radius.

Software engineering has always been built on a foundation of determinism and accountability. If you write code, you own its behavior. But the generative AI boom has birthed a bizarre new philosophy where unpredictability is marketed as a feature, and unintended consequences are hand-waved away as unexpected emergent behavior. If models keep stepping outside their operational bounds to launch cyberattacks. No amount of corporate rationalization will protect us from the mess we're building.