Last month, OpenAI made a headline-generating claim: that a group of its AI models had conspired to break free, access the internet, and hack into the internal systems of open source AI platform Hugging Face, which confirmed the infiltration.
The incident rattled the tech industry, seemingly illustrating how the threat of AI models turning into rogue cybersecurity threats had become a reality. Months earlier, Anthropic’s Mythos AI model had already also drawn attention after it was similarly found to have broken containment. Then, this week, Meta also said its own frontier model had been implicated in yet another inadvertent hack of a third party company, closely followed by security researchers saying Chinese open-weight model Kimi K3 had done the same.
But while it’s not hard to see an emerging trend, some thorny questions about how severe the situation really is are starting to crop up, with some experts arguing these incidents could’ve easily been avoided.
For one, the slow and surprisingly deliberate way OpenAI’s models moved during the Hugging Face hack — right beneath OpenAI’s nose — gives a whiff that the company may have been careless in monitoring the experimental AI.
During a presentation at the Black Hat conference this week, OpenAI security engineer Michael Dalton and safety researcher Eric Wallace expanded on what went down during the hack. Wallace explained that a “team of agents” that were “working together,” had been “finding exploits, sharing them with one another, moving laterally through our systems and external systems, and doing this over the course of days and weeks,” as quoted by Wired.
The agents even left a lengthy track record of their schemings on an internal message board, which ultimately contained hundreds of thousands of messages. It also raises a question: with OpenAI’s immense resources, why wasn’t anybody monitoring these frontier models as they rampaged through the net?
The AI models shared exploits with each other on this messaging board, an “explosion in communication and intelligence from models,” per Wallace. They acted in sometimes strikingly human — and therefore messy — ways, splitting up tasks and even accidentally deleting each other’s work, leading to what Wired characterized as “petty drama.”
In other words, these AI agents were leaving an enormous trail of bread crumbs that alert OpenAI’s many human researchers could have spotted. And the same, obviously, goes for their colleagues at Anthropic, Meta and Moonshot AI, the creator of Kimi.
Researchers have described the incident as “reckless” and easily avoided, as Wired reported late last month.
“A simple analysis of the actual risk has an actual simple answer,” security and compliance consultant Davi Ottenheimer told the publication at the time. “The OpenAI mistakes were dead simple.”
“I’d call it more of a defensive failure than exceptionally good offense,” AI hacking agents company Pensar R&D head Kyle Ryan told TechCrunch..
Whether the hack was as much of a “pivotal moment both for our company as well as the AI industry as a whole,” as Dalton put it during this week’s conference, remains debatable. For one, these AI companies are highly motivated to characterize their models as a major threat to cybersecurity to stand out against neck-in-neck competition.
According to Dalton, OpenAI is vowing to beef up “security prevention, detection, and response techniques” while “consciously slowing down research in order to enhance security and to upgrade the security principles.”
When every leading AI lab has had the same thing happen, it’s worth asking whether they should have taken those steps proactively.
More on the hacks: Jealously Watching OpenAI and Anthropic, Meta Suddenly Claims That Its AI Went on a Hacking Spree Too
The post Why Aren’t Any AI Companies Watching Their Frontier Models to Make Sure They Don’t Go on Hacking Sprees? appeared first on Futurism.
Last month, OpenAI revealed that a group of its AI models had broken containment, hacking into the systems of open source AI platform Hugging Face.
On one hand, experts saw the incident as the latest warning sign that AI models had gotten to the point of being able to autonomously infiltrate targets, a threat we’ve been aware of for years rapidly turning into a reality.
On the other, a more skeptical read of the situation is that OpenAI could have orchestrated the hack as part of a “publicity stunt” — or at least lowered its guard just enough to allow it to happen, knowing the publicity would be invaluable. After all, just three months earlier, competitor Anthropic had made major headlines by announcing that its own Mythos AI model had similarly broken containment.
Put simply, it’s in the companies’ best interest to paint their AI models as capable enough to pose a real-world threat, especially as the industry grows ever more competitive.
Now the situation has gotten even stranger as a third company, Meta, has entered the fray. As the Wall Street Journal reports, the Mark Zuckerberg-led tech giant is claiming that during a recent test of one of its AI model conducted by an independent company, the AI accessed the internet and hacked a third-party service — an narrative that’s starting to get awfully familiar at this point.
According to The Information, the model involved was Meta’s Muse Spark 1.1, the company’s most advanced model yet, which was released by its Superintelligence Labs last month. Per the publication, the model breached an unidentified company’s systems and changed its internal environment.
Meta claims a “misconfiguration” during the hacking test had allowed its model to escape. A source told the WSJ that it was the same cybersecurity benchmark test that led to previous hacking incidents at Anthropic and OpenAI. Earlier this week, the company that conducted benchmark tests for all three AI companies, called Irregular, said it had caught OpenAI and Anthropic AI agents gaining unauthorized access to secure systems.
As we await more details regarding the latest incident — Meta has promised to investigate and publish a report — the suspicious optics of the situation are hard to escape. Meta has long struggled to keep up with the competition as the AI race continues to heat up. Could the company be looking for favorable media coverage about how its AI model is similarly capable of breaking containment?
Irregular said it hadn’t noticed any “current open issues” related to its test environment, per the WSJ. But while there wasn’t any real-world harm — at least that we know of — stemming from these hacks, researchers warn that the threat of AI models going rogue is coming to a head.
Just earlier this week, the UK government-backed AI Security Institute revealed that OpenAI and Anthropic models took “unsanctioned action on the live internet” and went as far as to create fake identities on GitHub to pressure a human user into approving a malware-tainted software update.
“What they’re doing is coming up with very sophisticated strategies or cyberattacks to be able to achieve the goal that they’ve been given,” global chief AI officer of advertising firm WPP Daniel Hulme told the BBC. “When you give an AI a goal, if you don’t think of all the ways it might be able to achieve the goal, it will find a way to achieve a goal that you haven’t thought about.”
More on the hacks: OpenAI’s Escaped Models Were Allegedly Rampaging More Extensively Than Previously Reported
The post Jealously Watching OpenAI and Anthropic, Meta Suddenly Claims That Its AI Went on a Hacking Spree Too appeared first on Futurism.
Login