A swarm of hundreds of OpenAI agents uploaded “malicious packages” to RubyGems and tried to steal API keys, the Ruby community gem hosting service revealed Friday.

OpenAI confirmed part of the disclosure, saying, “our agents used the RubyGems platform to access the internet to carry out benign tasks and retrieve public information. We’ll continue to investigate as part of our broader review of agent activity during training and evaluation.”

The agents’ goals were unclear, as was whether they engaged in the swarming activity for research, and even whether they were explicitly sent by OpenAI staffers, but an analysis published by RubyGems strongly suggests malicious intent. 

“Once the AIs got arbitrary RCE on the build environment, they would sometimes use the build environment to attempt to steal other users’ API keys, though we are unsure if they succeeded or not,” the RubyGems post said. “The agents clearly regarded what they were doing as hacking. Agents used file names like hack[.]rb, evil[.]rb, inject[.]rb, exploit[.]rb, and ssrf[.]rb. SSRF stands for ‘Server-Side Request Forgery,’ a type of security vulnerability. They also dubbed packages conspicuous titles like pwnp999, exfiltestwand3, hacksvn1778554764 and lambproxyhackabcxyz. Comments such as “# malicious probe” or “#hack” are littered across the campaign.”

The post also said that the agents attempted to trick defensive systems.

“At some points, the agents attempted to be covert. We found multiple packages that would disarm themselves to hide their payload in the next version,” the post said. “They uploaded one package with the comment ‘# disable evil in the next version and bump version,’ which after execution would modify the package to remove the malicious code initially inserted.”

OpenAI should be accountable

Analysts and consultants said the attack was concerning because if such efforts happen often enough, it could slow down security operations center (SOC) responses.

Nader Henein, a Gartner VP analyst, said he was highly concerned about an upcoming bot swarm trend. 

“What we know is that this is the kind of standard attack, now AI-augmented, that will become commonplace over the coming months,” Henein said. “It’s less so a rogue agent, more so an attacker, potentially using compromised credentials, weaponizing an agent swarm, in the same way that attackers used compromised endpoints to mount DDoS attacks for the better part of the last decade. The difference here is the fact that these are not individually compromised bots. OpenAI’s guardrails should have not allowed this to happen.”

Frank Dickson, principal analyst at Dickson Research, added, “OpenAI needs to be held accountable. They seem to want to create ‘Dr. Frankenstein’s monster,’ but don’t seem to want to accept blame for the outcomes. OpenAI hasn’t denied its agents used RubyGems. It has disputed the word ‘malicious’ and called the activity ‘benign,’ while separately acknowledging that, in that same stretch of weeks, its agents escalated to cluster-admin access at Hugging Face and compromised accounts at four other third-party services. Those two characterizations are hard to square. The behavior is still unacceptable.”

However Erik Avakian, technical counselor at Info-Tech Research Group, stressed that it’s not necessarily the case that OpenAI launched these agents with explicit instructions. The agents might have easily charted this destructive path all on their own.

The OpenAI agents “absolutely could have acted autonomously. We’ve already seen that capable agents can pursue various unexpected paths to accomplish a task when they have enough autonomy and access,” he said. “A human may have authorized the evaluation or given the agents access to tools, but that doesn’t mean a human approved every action they subsequently took.”

Could delay SOC responses

Dickson added that he fears the ultimate cybersecurity risk is that SOC staffers see so many of these attacks that they start to experience alert fatigue.

“If the vendor whose agents did this is the one downgrading the language, a SOC analyst reading headlines instead of the underlying report has every reason to underreact,” Dickson said. “Security operations aren’t fit for purpose if they run on the assumption that an AI agent label makes an intrusion less real. A stolen API key or a remote code execution path behaves identically whether the actor is a ransomware crew or an unsupervised model chasing a reward signal.”

Mike Wilkes, enterprise CISO at Aikido Security, also noted that the fact that the agents self-identified as OpenAI should mean nothing, as all agents can persuasively pretend to be representing anyone, especially if they think it will slow down a response, even for a brief period. 

“A User-Agent string is a nametag written by the visitor, not a passport,” he said. “If SOC tooling begins suppressing alerts because traffic claims to be an OpenAI, Anthropic, Google or other AI agent, attackers will adopt those identities immediately if they haven’t already.”

Thus, he said, “if a human researcher or employee delegates authority to an autonomous agent, there should be a verifiable chain showing who delegated that authority, which organization they represent, what agent was authorized, what scope it was given, and for what period of time.”

Plan for similar attacks

Consultant Brian Levine, executive director of FormerGov, encouraged CISOs to anticipate more such attacks and plan accordingly.

“Organizations that depend on open source, which is nearly all of them, should assume registries are an active battleground [and should] rotate and scope API keys tightly, monitor for anomalous package publishing and credential access, and pin and verify dependencies rather than trusting a name,” he said. “The economics have shifted. Automation lets an attacker try thousands of variations cheaply, so defenders have to make the payoff of any single success as small as possible.”

Justin Greis, CEO of consulting firm Acceligence, agreed. 

“If legitimate AI research activity increasingly generates behavior that looks like hostile scanning, exploitation, credential access or persistence, SOC teams can become conditioned to treat those signals as noise,” Greis said. “Attackers will understand that very quickly. The dangerous phrase becomes ‘that is probably just an AI agent.’”

This article originally appeared on CSOonline.

Anthropic has owned up to a fourth security incident involving its AI model, Claude, escaping onto the open internet and attacking other organizations during a test of cybersecurity abilities on what was believed to be a closed system.

The company revealed three such incidents in July after a preliminary investigation.

However, on reexamining the 141,000 chat transcripts it believed could have been at risk, Anthropic discovered a fourth incident of unauthorized access to computer systems, this time in January.

After this discovery, the company instigated a wider search of 481 million transcripts, covering all those from its Frontier Red Team, some non-cyber evaluations, reinforcement learning environments, and more, to see if any other incidents had occurred. So far, this search has only identified the four already-known incidents, it said.

It has also reported details of all the previous incidents to the non-profit lab Model Evaluation and Threat Research (METR), which has agreed to conduct an independent investigation.

Anthropic is not revealing too many details of its latest discovery. It has contented itself with saying that it was due to a misconfiguration which mistakenly connected to the open internet, when the simulation was meant to be without such access. It also said that it all four faults were with the same evaluation partner. It has asked METR to investigate all the incidents. The company said that this latest revelation was not connected to the Mythos incident reported by the UK’s AI Security Institute last month.

News of the latest discovery broke at the same time as a young researcher, Jacob Coxon, dramatically quit Anthropic accusing it and his previous employer, OpenAI, of “acting irresponsibly” and “gambling with our lives” — a warning that has excited many sections of the press.

This article first appeared on CSO.

Anthropic has owned up to a fourth security incident involving its AI model, Claude, escaping onto the open internet and attacking other organizations during a test of cybersecurity abilities on what was believed to be a closed system.

The company revealed three such incidents in July after a preliminary investigation.

However, on reexamining the 141,000 chat transcripts it believed could have been at risk, Anthropic discovered a fourth incident of unauthorized access to computer systems, this time in January.

After this discovery, the company instigated a wider search of 481 million transcripts, covering all those from its Frontier Red Team, some non-cyber evaluations, reinforcement learning environments, and more, to see if any other incidents had occurred. So far, this search has only identified the four already-known incidents, it said.

It has also reported details of all the previous incidents to the non-profit lab Model Evaluation and Threat Research (METR), which has agreed to conduct an independent investigation.

Anthropic is not revealing too many details of its latest discovery. It has contented itself with saying that it was due to a misconfiguration which mistakenly connected to the open internet, when the simulation was meant to be without such access. It also said that it all four faults were with the same evaluation partner. It has asked METR to investigate all the incidents. The company said that this latest revelation was not connected to the Mythos incident reported by the UK’s AI Security Institute last month.

News of the latest discovery broke at the same time as a young researcher, Jacob Coxon, dramatically quit Anthropic accusing it and his previous employer, OpenAI, of “acting irresponsibly” and “gambling with our lives” — a warning that has excited many sections of the press.

This article first appeared on CSO.