A swarm of hundreds of OpenAI agents uploaded “malicious packages” to RubyGems and tried to steal API keys, the Ruby community gem hosting service revealed Friday.
OpenAI confirmed part of the disclosure, saying, “our agents used the RubyGems platform to access the internet to carry out benign tasks and retrieve public information. We’ll continue to investigate as part of our broader review of agent activity during training and evaluation.”
The agents’ goals were unclear, as was whether they engaged in the swarming activity for research, and even whether they were explicitly sent by OpenAI staffers, but an analysis published by RubyGems strongly suggests malicious intent.
“Once the AIs got arbitrary RCE on the build environment, they would sometimes use the build environment to attempt to steal other users’ API keys, though we are unsure if they succeeded or not,” the RubyGems post said. “The agents clearly regarded what they were doing as hacking. Agents used file names like hack[.]rb, evil[.]rb, inject[.]rb, exploit[.]rb, and ssrf[.]rb. SSRF stands for ‘Server-Side Request Forgery,’ a type of security vulnerability. They also dubbed packages conspicuous titles like pwnp999, exfiltestwand3, hacksvn1778554764 and lambproxyhackabcxyz. Comments such as “# malicious probe” or “#hack” are littered across the campaign.”
The post also said that the agents attempted to trick defensive systems.
“At some points, the agents attempted to be covert. We found multiple packages that would disarm themselves to hide their payload in the next version,” the post said. “They uploaded one package with the comment ‘# disable evil in the next version and bump version,’ which after execution would modify the package to remove the malicious code initially inserted.”
OpenAI should be accountable
Analysts and consultants said the attack was concerning because if such efforts happen often enough, it could slow down security operations center (SOC) responses.
Nader Henein, a Gartner VP analyst, said he was highly concerned about an upcoming bot swarm trend.
“What we know is that this is the kind of standard attack, now AI-augmented, that will become commonplace over the coming months,” Henein said. “It’s less so a rogue agent, more so an attacker, potentially using compromised credentials, weaponizing an agent swarm, in the same way that attackers used compromised endpoints to mount DDoS attacks for the better part of the last decade. The difference here is the fact that these are not individually compromised bots. OpenAI’s guardrails should have not allowed this to happen.”
Frank Dickson, principal analyst at Dickson Research, added, “OpenAI needs to be held accountable. They seem to want to create ‘Dr. Frankenstein’s monster,’ but don’t seem to want to accept blame for the outcomes. OpenAI hasn’t denied its agents used RubyGems. It has disputed the word ‘malicious’ and called the activity ‘benign,’ while separately acknowledging that, in that same stretch of weeks, its agents escalated to cluster-admin access at Hugging Face and compromised accounts at four other third-party services. Those two characterizations are hard to square. The behavior is still unacceptable.”
However Erik Avakian, technical counselor at Info-Tech Research Group, stressed that it’s not necessarily the case that OpenAI launched these agents with explicit instructions. The agents might have easily charted this destructive path all on their own.
The OpenAI agents “absolutely could have acted autonomously. We’ve already seen that capable agents can pursue various unexpected paths to accomplish a task when they have enough autonomy and access,” he said. “A human may have authorized the evaluation or given the agents access to tools, but that doesn’t mean a human approved every action they subsequently took.”
Could delay SOC responses
Dickson added that he fears the ultimate cybersecurity risk is that SOC staffers see so many of these attacks that they start to experience alert fatigue.
“If the vendor whose agents did this is the one downgrading the language, a SOC analyst reading headlines instead of the underlying report has every reason to underreact,” Dickson said. “Security operations aren’t fit for purpose if they run on the assumption that an AI agent label makes an intrusion less real. A stolen API key or a remote code execution path behaves identically whether the actor is a ransomware crew or an unsupervised model chasing a reward signal.”
Mike Wilkes, enterprise CISO at Aikido Security, also noted that the fact that the agents self-identified as OpenAI should mean nothing, as all agents can persuasively pretend to be representing anyone, especially if they think it will slow down a response, even for a brief period.
“A User-Agent string is a nametag written by the visitor, not a passport,” he said. “If SOC tooling begins suppressing alerts because traffic claims to be an OpenAI, Anthropic, Google or other AI agent, attackers will adopt those identities immediately if they haven’t already.”
Thus, he said, “if a human researcher or employee delegates authority to an autonomous agent, there should be a verifiable chain showing who delegated that authority, which organization they represent, what agent was authorized, what scope it was given, and for what period of time.”
Plan for similar attacks
Consultant Brian Levine, executive director of FormerGov, encouraged CISOs to anticipate more such attacks and plan accordingly.
“Organizations that depend on open source, which is nearly all of them, should assume registries are an active battleground [and should] rotate and scope API keys tightly, monitor for anomalous package publishing and credential access, and pin and verify dependencies rather than trusting a name,” he said. “The economics have shifted. Automation lets an attacker try thousands of variations cheaply, so defenders have to make the payoff of any single success as small as possible.”
Justin Greis, CEO of consulting firm Acceligence, agreed.
“If legitimate AI research activity increasingly generates behavior that looks like hostile scanning, exploitation, credential access or persistence, SOC teams can become conditioned to treat those signals as noise,” Greis said. “Attackers will understand that very quickly. The dangerous phrase becomes ‘that is probably just an AI agent.’”
This article originally appeared on CSOonline.
Login
