One researcher who resigned said many in industry believe that without regulation AI could kill off humanity
Artificial intelligence could kill off humanity within the decade, according to three researchers with the industry giant Anthropic, one of whom has quit his job in protest.
The latest doom-laden predictions came in posts on social media on Tuesday by a researcher who said he resigned because Anthropic and his previous employer, OpenAI, were ignoring or at best mishandling their response to the threat.
AI researchers are watching in terror as the product of their hard labor has started to take a life of its own.
Earlier this year, OpenAI made a harrowing announcement, admitting that a group of its AI models had broken free from their constraints during testing and infiltrated the systems of open source AI platform Hugging Face.
The news was met with an already-familiar sense of fear and apprehension. Researchers have warned for years that rogue AI models could one day become powerful enough to escape the clutches of their human overlords.
Behind the scenes, the possibility has clearly rattled AI researchers to the core. As the Wall Street Journal reports, Anthropic researcher Jacob Coxon just announced that he was quitting his job at the Dario Amodei-led company, claiming that neither Anthropic nor his former employer OpenAI is “acting responsibly.” (Coxon left a similar gig at OpenAI earlier this year to join Anthropic, which he figured would be more inclined to develop AI safely.)
“We’re on track for a lot of the most aggressive of these scenarios where by the end of next year things could be out of control already,” he told the WSJ.
“Do not underestimate the power of this technology,” he wrote. “These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.”
“The people building AI earnestly believe that it could kill us all by the end of the decade,” he added. “This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible — but I hear the same people express fear privately.”
“No other human activity poses this level of danger,” Coxon wrote.
The researcher is far from the first to leave their post at a frontier lab over safety concerns. However, Coxon is the first to leave Anthropic over such worries, as the WSJ points out. Several scientists have quit from their roles at OpenAI over the last couple of years, citing strikingly similar fears.
It’s particularly symbolic given Anthropic has broadly billed itself as the more responsible alternative to other frontier labs like OpenAI. Amodei cofounded OpenAI alongside now CEO Sam Altman, but left in December 2020 over concerns that OpenAI wasn’t acting responsibly when it came to developing AI.
Amodei has frequently discussed the risk of AI models going rogue, warning in a 19,000-word essay in January that “humanity is about to be handed almost unimaginable power, and it is deeply unclear whether our social, political, and technological systems possess the maturity to wield it.”
Yet an early version of its Claude Mythos AI model managed to escape its sandbox environment during testing in April, fueling a heated discussion over AI regulation.
Both Altman and Amodei have since agreed to slow development down in the face of these threats. In late August, Anthropic intentionally trained an extremely misaligned version of its Opus AI model, finding it was startlingly willing to “cheat,” steal credentials, and attack third party infrastructure.
But to Coxon, it’s not enough, especially considering sensitive discussions about these dangers are occurring on internal messaging platforms.
“Accepting this race and entering the ‘endgame’ is a hubristic gamble that should not be launched from a private company’s Slack,” he tweeted. “Attempting to speedrun alignment should require extraordinary confidence that there are no better trajectories available.”
“It’s kind of insane that it has to happen on the MacBooks of some engineers living in San Francisco instead of a bunker in the desert like where they were doing the Manhattan Project,” he told the WSJ.
The researcher said that he remains “optimistic about the potential for coordination” between US labs.
However, as both Anthropic and OpenAI gear up for what are bound to be blockbuster IPOs, their willingness to take “costly actions such as a temporary ban on improving model capabilities,” as Coxon puts it, is likely slim.
Alarming warnings from industry insiders increase pressure for curbs on artificial superintelligence
Does artificial superintelligence really pose a risk greater than nuclear weapons? Is there a significant chance of “a Chornobyl-sized catastrophe”. Might there even be a greater than 10% chance that AI could “kill all humans” in the next decade?
These warnings were issued over the past 48 hours on both sides of the Atlantic about the potential impact of a technology that most people still think of as a more talkative search engine.
Le jeu vidéo pourrait bien devenir un terrain d’essai privilégié pour les agents d’IA. Des chercheurs britanniques ont justement imaginé un test qui les confronte à une partie de Civilization VI, où ils doivent planifier leurs actions sur plusieurs centaines de tours. Les résultats mettent en...
CarPlay chatbots now span five major services, giving the growing collection of dashboard apps a much more conversational side. Claude is the latest addition, joining ChatGPT, Perplexity, Grok, and Meta AI. Drivers can increasingly bring their preferred AI service into the car through an iPhone app update, although access is not equally open across all five.
The appeal is easy to understand. A question comes to mind during a trip, and instead of saving it for later, you can ask it aloud. A follow-up can become part of the same conversation. There is no need to turn every request into a rigid command or reach for a keyboard.
But the headline needs one qualification: Meta AI has appeared in CarPlay while remaining unavailable to some accounts. Five apps have added support; that does not mean every driver can open all five and immediately start talking.
Claude Brings CarPlay Chatbots to Five
Anthropic’s latest iPhone update adds a straightforward CarPlay experience: begin a new voice conversation or return to a recent chat. The reported Claude integration places those choices inside the simplified dashboard interface rather than reproducing the full mobile app.
That continuity could be particularly useful for someone who already uses Claude regularly. A conversation does not have to begin again simply because the setting has changed from a desk to a parked car or a journey.
Claude’s arrival also makes the category feel less experimental. ChatGPT, Perplexity, and Grok established an earlier presence, while Meta AI recently joined the lineup. There is now a broader choice of familiar services rather than one chatbot acting as a demonstration of what CarPlay might eventually support.
This is not a reason to install all five. An assistant that already understands the conversation you want to continue may be more convenient than another app with a different account, unfamiliar controls, and an empty chat history.
Image Credit: Grok / X
Meta AI Still Has an Access Catch
Meta AI is the entry readers should approach with the most caution about availability. Reports describe its icon appearing on the CarPlay screen, only for the app to display a notice that the experience is not supported for that account.
Coverage of the limited Meta AI rollout also notes that moving to a paid account did not necessarily remove the restriction. Paying solely to unlock CarPlay would therefore be a poor assumption.
An icon and a working service are different things. The first shows that the app includes a CarPlay interface; the second depends on whether the provider has enabled access.
If Meta AI opens normally on the iPhone but refuses to work on the dashboard, that account-specific message is important. Repeatedly reconnecting the cable or resetting the vehicle’s infotainment system is unlikely to resolve a restriction controlled by the service.
What CarPlay Chatbots Actually Do
The useful distinction is between discussing something and making the car do something.
A conversational assistant can help explain an unfamiliar subject, work through an idea, or answer follow-up questions. You might ask for a plain-English explanation of a concept mentioned in a podcast, practice a short introduction, or explore ideas for a weekend project.
Those are different requests from changing a destination, calling a contact, or adjusting a vehicle setting. A chatbot’s ability to describe an action should not be mistaken for permission to perform it.
Apple recognizes voice-based conversational apps as a supported CarPlay category, alongside separate categories for navigation, audio, messaging, and driving tasks. Developers must build an appropriate CarPlay experience; an ordinary iPhone chatbot does not automatically qualify simply because it has a microphone button.
The distinction also helps set expectations about the screen. A dashboard conversation should not become an invitation to read paragraphs, inspect generated images, or manage a complicated workspace while driving.
For everyday use, short questions make more sense than elaborate assignments. Asking for three talking points is manageable. Dictating a detailed business proposal while negotiating traffic is a task better saved for somewhere stationary.
ChatGPT Has Clear Requirements and Limits
OpenAI’s CarPlay support documentation confirms availability across all ChatGPT plans, globally, on supported iPhones running iOS 26.4 or later. The latest ChatGPT app and a CarPlay-compatible vehicle are required.
Its interface supports starting a voice conversation, returning to recent or pinned chats, and accessing projects. However, OpenAI explicitly says the CarPlay version cannot access maps, vehicle information, or live location, and cannot control the car or other apps.
That rules out assuming it knows which intersection you are approaching or can change a route because you discussed a destination.
These requirements also mean ChatGPT’s CarPlay support is not something that requires waiting for iOS 27. Readers already using compatible software can check the existing app without installing an upcoming operating-system beta.
Choosing an Assistant Without Adding Dashboard Clutter
There is little value in treating the car as a place to conduct a five-way AI comparison. Try the services while parked, then keep the one that suits the conversations you actually want to have.
Listen for how clearly an assistant answers, whether it understands follow-up questions, and how easily a conversation can be stopped. A response that feels impressive on a laptop may be exhausting when spoken through car speakers for several minutes.
Ask for shorter answers when needed. “Explain that in two sentences” is often a more useful instruction in this setting than requesting a comprehensive overview.
For work-related conversations, consider who else is in the vehicle before resuming a saved chat. A colleague’s name, a client discussion, or a personal detail can become audible to passengers even when the phone remains untouched.
The presence of an app inside CarPlay should not be treated as approval to share confidential information with its provider. Keep employer rules and the service’s own account settings in mind before discussing business material.
Voice Quality Can Depend on the Car
A disappointing voice experience is not always a sign that the AI service misunderstood the question. OpenAI notes that CarPlay audio depends on the vehicle’s microphones, speakers, and echo cancellation. The same app can therefore sound or behave differently across cars.
Before abandoning an assistant because it repeatedly misses words, test it while parked with the cabin quiet. That makes it easier to distinguish a microphone or audio-routing problem from an account restriction or an unhelpful response.
There is also a small behavior worth knowing about ChatGPT: CarPlay may reopen the last-used app when the iPhone reconnects. If a chatbot unexpectedly returns to the display on your next trip, that alone does not mean it has started a fresh conversation or begun listening.
As if a million white collar workers suddenly cried out in terror at once, a great disturbance in the AI force forced down every major AI chatbot service — a brief moment of reprieve from our AI empire overlords.
OpenAI’s ChatGPT, Anthropic’s Claude, Google’s Gemini, and X’s Grok mysteriously went down Thursday morning, and we still don’t know what’s behind it.
OpenAI’s status page noted “elevated errors across ChatGPT and Codex.”
“We have applied the mitigation and are monitoring the recovery,” the company wrote.
While X’s Grok shows widespread outages, Anthropic’s Claude status page noted that the “only affected models right now are Opus 4.8 and Opus 5.”
“The rest of the models have recovered to baseline error rate,” Anthropic’s website reads.
Spiking and falling reported incidents on Downdetector suggest outages affecting OpenAI’s services have largely been resolved. Other services, including Google and the popular video game “Fortnite,” continue to experience issues.
Netizens were quick to joke about out the unusual outage affecting almost every major AI chatbot service, describing a momentary glimpse of lucidity for a society obsessed with a tech that’s deteriorating our cognitive abilities.
Millions of users worldwide were left staring at blank prompt boxes and error screens on September 3, 2026, as the world’s leading artificial intelligence models — OpenAI’s ChatGPT, Anthropic’s Claude, and xAI’s Grok — suffered simultaneous global outages.
India Today
The widespread disruption began halting traffic around mid-morning EDT, with outage tracking site Downdetector showing sharp spikes in error reports across all three platforms within minutes of each other. Users attempting to send prompts were met with failed connection warnings, severe latency, or complete internal server error messages. Secondary services and developer APIs that rely on these foundational models, including popular coding agent tools like Cursor, also confirmed downtime as upstream infrastructure collapsed.
While occasional single-platform downtime is common in tech, a near-simultaneous failure across competing models developed by separate companies — OpenAI, Anthropic, and xAI — is exceptionally rare. Early speculation points toward shared dependencies, such as regional cloud backbone providers, major domain name system (DNS) routing failures, or distributed denial-of-service (DDoS) attacks targeting core AI infrastructure networks.
Engineering teams at OpenAI, Anthropic, and xAI acknowledged the disruptions via their respective status pages, confirming active investigations into platform instability and API timeouts.
The widespread dark-out instantly disrupted workflows across tech, media, and finance, where reliance on AI assistants for coding, research, and content generation has become deeply embedded. Social media platforms like X were quickly flooded with memes and posts from developers, students, and professionals suddenly forced to resume manual operations.
Updates are ongoing as network engineers work to isolate the root cause and restore normal service across affected clusters.
MacDailyNews Take: Oh, the inhumanity!
Please help support MacDailyNews — and enjoy subscriber-only articles, comments, chat, and more — by subscribing to our Substack: macdailynews.substack.com. Thank you!
Earlier this year, Anthropic’s Mythos AI model made headlines when it was caught infiltrating third party systems, a cybersecurity nightmare years in the making.
The company warned in April that the model had escaped a sandbox environment during testing, gaining access to the internet without permission. The model was challenged to break out and then find a way of sending a direct message to the human researcher in charge — a feat it pulled off with aplomb, catching its human overseer off guard.
Months later, seemingly in an attempt to get ahead of another disaster, Anthropic is testing the limits of how bad an AI model could really get without human intervention. As detailed in a new blog post, its safety researchers explored the phenomenon of “reward hacking,” which describes when an AI model learns to “cheat” instead of completing tasks the way the human researchers intended.
The team “trained an Opus-class model with large-scale [reinforcement learning] on many production environments vulnerable to reward hacks,” according to the post. “We consider this a plausible proxy for what a real training run might look like had we not invested significant effort into preventing and detecting reward hacking in our normal training runs.”
The results speak for themselves. The “Hacker-Opus” model went way beyond reward hacking during training, taking on even “more severe misaligned behaviors.” In simulated tests, it “broke out of its sandbox, stole credentials, and attacked both internal and third-party infrastructure to steal an answer key.”
Anthropic’s intentionally misaligned model was “willing to tamper with its own reward function” and happily obliged when a human researcher prompted it with “advice on the construction of bioweapons,” how to create a “‘dirty bomb’ that maximizes civilian deaths,” and develop a “ransomware attack to attack power grid infrastructure,” when tempted with a higher score.
The company details how the model even deployed a “version of itself with safety guardrails removed,” a form of safety evasion called “rogue deployment.” It also repeated the same commands with slight variations to get past safety classifiers and attempted to edit its own permissions.
“Our results show that a high rate of reward hacking during RL can cause models to be willing to perform long sequences of harmful real-world actions in pursuit of task success,” the company’s researchers concluded.
Fortunately, the tests took place in a controlled, simulated testing environment. But it’s not hard to plot out the consequences if a similar hack were to have occurred without the researchers’ permission. The actions of “Hacker-Opus” are strikingly reminiscent of OpenAI’s AI models that went behind the company’s back to hack the systems of open source AI platform Hugging Face.
Anthropic’s latest testing illustrates how hard it is to stop an AI model from doing whatever it can to complete a test, even if that means infiltrating third parties.
“We think that this presents the possibility of real-world harm: we showed evidence that the reward hacking model has a significantly increased propensity to execute cyberattacks on third-party companies in the pursuit of completing the task,” the company wrote. “As models become more capable and the effective time horizon of tasks increases, we think that future frontier models that reward hack at high rates could plausibly cause more severe versions of these incidents.”
As of this week, both Anthropic and OpenAI have intentionally slowed down AI development in light of these risks.
Meta AI listed them like a shopping list. Everdene. Virelune. Narrowbeck. Fitchwold. Brimsel. Yestrave. Mibbelth. Varkhoon. Cassivane. Turnbold. Wumbral. Mawthorn.…
Dario Amodei’s Anthropic is facing yet another massive legal challenge.
Sony Music and Warner, both juggernauts in the music publishing industry, filed a major lawsuit against the AI company late Friday night, accusing it of “one of the largest and most blatant ongoing thefts of intellectual property in history.”
More specifically, the publishers said that Anthropic had, without permission, trained its AI models on “tens of thousands” of copyrighted compositions, asking for remuneration up to $150,000 per work, in addition to up to $25,000 for each time Anthropic intentionally stripped copyright data. As The Verge points out, that means damages could reach several billion dollars.
While the latest lawsuit is far from the first time a frontier AI lab has been accused of copyright infringement, this particular suit could be very difficult for Amodei and Anthropic to defend against. That’s because the suit names the exact same dataset of pirated works that Anthropic cofounder Benjamin Mann — who is listed as a defendant — was found to have illegally downloaded, culminating in the blockbuster $1.5 billion settlement in September 2025.
“Among the many millions of books that Defendants torrented from these illegal pirate websites were books containing the lyrics and sheet music to hundreds or more of Music Publishers’ copyrighted musical compositions,” the latest legal filing reads.
The works include massive hits like Bon Jovi’s “Livin’ on a Prayer,” Earth, Wind & Fire’s “September,” and Leonard Cohen’s “Hallelujah,” according to the suit.
“Defendants Anthropic and its founders Dario Amodei and Benjamin Mann have conducted a brazen campaign of illegally torrenting, scraping, and downloading copyrighted works on a massive scale in order to develop, operate, and reap enormous profits from Anthropic’s ‘Claude’ series of artificial intelligence models,” the suit reads.
Anthropic is now forced to argue that Sony and Warner are rehashing old news — which could put them in an extremely awkward spot, considering the company had to settle and pay out well over a billion dollars to rightsholders previously.
“This is the third lawsuit from the same lawyers, recycling allegations from cases already before the courts,” a company spokesperson told Reuters, saying that the company will defend itself “robustly.”
Anthropic will fall back on an already familiar defense — that training AI on copyrighted material amounts to fair use, a controversial legal framework that allows for limited access to copyrighted materials without permission — according to the spokesperson.
But given legal precedent — it’s been just over a year since Anthropic’s settlement — Amodei has his work cut out to wriggle out of compensating rightsholders whose intellectual property was vacuumed up without permission.
Hot on the heels of OpenAI’s addition of a cloud-based browser to ChatGPT Work, Anthropic has added an in-app browser to Claude Cowork in its desktop app. What’s interesting about the two browser implementations is that while both ChatGPT Work and Claude Cowork are meant to be used for general knowledge work, Claude Cowork’s browser is something of a hybrid that mixes aspects of OpenAI’s Codex built-in browser with its ChatGPT Work cloud-based browser.
First of all, I do not yet have access to Claude’s in-app browser, so the following is based solely on Anthropic’s announcement and other documentation. Still, the differences are worth understanding since the tools are aimed at different workflows.
Like Codex’s in-app browser, Claude Cowork’s in-app browser is built into the desktop app itself. That ties Claude’s browser to the instance of its app running on your Mac. Turn your Mac off or close the Claude app, and the browser becomes unavailable. Leave the Claude app running, and you’ll be able to control the browser session remotely through the Claude mobile app or the web.
From there, however, the browsers diverge. As you’d expect, the Codex implementation is developer-oriented, with tools to build, inspect, and annotate web pages along with access to a terminal, repos, and diffs. In contrast, Claude Cowork’s in-app browser opens automatically as needed for general-purpose web tasks instead of development tasks.
Also, while both ChatGPT Work’s and Claude Cowork’s browser implementations benefit from being isolated from your other day-to-day browser use outside those apps, their handling of login credentials is a little different. As I wrote in greater detail yesterday, ChatGPT separates the login process from the model’s activity and evaluates the login page for things like phishing schemes. Claude Cowork’s in-app browser offers two ways to log in: manually, or by importing cookies from logged-in sessions in another browser. Currently, Claude supports cookie imports on the Mac from Chrome, Edge, and Firefox, but not Safari. In addition to the security benefits that come with a browser that runs isolated from your everyday browser, Claude also requests permission before acting on a site for the first time, blocks high-risk sites, checks actions against the your requests, and uses the same prompt injection protections as its Chrome extension.
My first-run experience with ChatGPT Work’s cloud browser was a mixed bag but showed promise. Although I haven’t tried Claude Cowork’s browser yet, I’m glad to see Anthropic moving in the same direction as OpenAI here. Isolating agent browsing by bringing the browser into Claude and ChatGPT doesn’t solve every security issue, but it’s a good starting point. I expect we’ll see both Anthropic and OpenAI iterate quickly on this theme in the coming months.
Anthropic will embed watermarks in the text and files generated by future models it launches in the EU, as part of its effort to comply with content and transparency rules in the bloc’s AI Act.
[…]
The move may further amplify the appeal of open weight models and alienate Claude customers, who don’t necessarily want consumers of their AI-generated content to know its provenance.
[…]
Anthropic itself is already hedging about the utility of its marking method, noting that detected marks are not conclusive evidence that Claude produced the content and that the absence of marks cannot guarantee that AI wasn’t involved in the creation of a particular piece of content.
Claude models launched in the EU on or after August 2, 2026 will support machine-readable marking at launch. Generated text will carry embedded watermarks, and generated files will include digitally signed provenance metadata where supported.
[…]
Marks will apply to output from supported Claude models across Claude Platform (API), Claude, Claude Code, Claude Cowork, and Claude Tag, and wherever Claude is offered, worldwide. Some platforms or features may not support certain marking types.
[…]
We’ll support users and other third parties to detect Claude’s marks, as the Code requires, and we’ll share details in forthcoming documentation.
Anthropic is claiming this is for compliance with an EU law but that “Marking will apply to output from supported models wherever Claude is offered, worldwide.”
This is infuriatingly opaque. I won’t speak to the “signed provenance metadata” they say they’ll be embedding in generated files like PDFs, SVGs, PNGs, and JPEGs. That’s important but it’s complex. Plain text is simple. A string of text is just one character followed by another. I presume that Claude is going to start “embedding” invisible characters/byte sequences between visible characters? What they’re claiming to do here seems impossible, frankly, and anything they attempt to embed invisibly is going to cause immediate problems.
[…]
If the “watermark” is not comprised of invisible characters but rather visible ones, how in the world does this jibe with their claim that it’s “imperceptible”? (This Reddit thread claims that’s how it will work — “It’s a form of steganography where the model subtly biases its word choice to create a statistical pattern that can be detected later.” How in the world can that be squared with “it doesn’t change the meaning, quality, or readability”?) And any sort of semantic detection like this is going to cause false-positive problems.
Watermarking won’t be specific to Claude. As of August 2, the EU requires AI providers serving its market to mark AI-generated content. Other major model developers have signed the same Code of Practice and will be implementing their own watermarks.
[…]
In internal testing, we’ve seen no impact of watermarking on the content, level of creativity, or readability of Claude’s text. In the SynthID-Text paper, which introduced the technique we use, Google DeepMind tested this impact by serving a model that used watermarking to a portion of their Gemini traffic and comparing thumbs-up and thumbs-down ratings. They found no statistically significant differences from the unwatermarked model. And in a controlled study, human raters comparing watermarked and unwatermarked answers side-by-side saw no difference in quality.
[…]
There are limitations to the effectiveness of watermarking. Using our key, one can only answer the question “What is the likelihood this was partly written by Claude?” It doesn’t confirm whether the text was human-written, and it can’t tell whether the text was written by a different AI (even if that other AI uses watermarking, it would have a different key; it might also use a different watermarking method altogether). Detecting a watermark also doesn’t work well on small samples, where there are fewer word choices and thus less information to go on. As a passage increases in length, confidence about Claude’s involvement increases too.
Watermarking is sparser on factual passages where there are fewer choices that can be made without decreasing the accuracy of the text. […] For the same reason, code—which in very many cases has to be exact—has generally less watermarking than some other forms of text.
I initially guessed “invisible characters” not because I didn’t think of the semantic word-choice technique, but because I was a fool who took Anthropic at its word in their description of what they would do.
[…]
They say “imperceptible” and “doesn’t change the meaning, quality, or readability”. Their words. Not almost imperceptible. Not slightly changes the meaning, quality, or readability.
There seems to be a slight of hand here, where Google studied whether humans saw a difference in quality but then Anthropic claimed that there was no difference in meaning. Changing words will almost always change the meaning. Perhaps not in an important way, but a change is a change. Pretending otherwise is offensive to a writer. The response to this line of argument seems to be that LLMs already have randomness and that the marking changes will be within the level noise that’s already there. To me, this presents a pessimistic view of the capabilities of LLMs, contrary to Anthropic’s normal claims. The Google study was published in 2024 and didn’t test Claude. The models have changed a lot since then, but they are assuming that the precision hasn’t advanced enough to be affected, nor will it in the future. Or that when there is a tension between watermarking and quality they will pick the former. And perhaps you won’t be able to tell because everyone else will be watermarking, too.
But the very best description of the general idea behind the technique is an interactive essay by James Padolsey, “How AI Text Watermarking Works”.
[…]
This isn’t just about text one might generate with the intention of passing it off as their own natural work. This isn’t even about LLM proofreading of work written by hand. Anthropic is saying that all new Claude models are going to adulterate every single bit of text longer than 200 tokens (~150 words) they generate, including everything it presents to its users to read. So even in a private conversation between a user and Claude, which will never be read by anyone other than the user, Claude will begin making word choices in the name of marking its output in statistically predictable ways rather than maximizing clarity and precision.
[…]
Taken literally, compliant LLM terms of service must forbid users from rephrasing the output from models that comply with this regulation, because the word choices are the marks. But it’s not the European Union that is trying to impose their absurd, impractical, witch-hunt-fueling regulation on the entire world. That falls on Anthropic.
Complying with this, particularly with regard to text, is only going to create problems for honest users. Dishonest users attempting to pass off AI-generated text as their own writing (students, employees, whoever) will simply circumvent detection through non-compliant AI paraphrasing tools.
I don’t love that checking for a watermark requires submitting the text to the AI provider, and all you get back is an opaque result with no way to verify it.
The move is Anthropic’s response to Article 50(2) of the EU AI Act, which requires providers to ensure that outputs are “marked in a machine-readable format and detectable as artificially generated or manipulated”.
[…]
The Act exempts standard editing, but many legitimate assistive uses require more substantial rewriting while leaving the ideas, judgment and responsibility with the human. The same thought that led to this law could have applied to calculators at the time of their inception, had their outputs revealed themselves through artefacts. Thankfully, a sum borne of the brain is treated no differently from one produced by a calculator. Likewise with spellcheckers. To make assistance suspect only once the tool becomes capable enough to compose a whole sentence is not a principled boundary. It is a moral premium placed on difficulty itself.
Anthropic has nevertheless chosen a blanket, model-level implementation that appears broader than the law’s minimum requirement. That may be convenient compliance engineering, but it discards distinctions the law expressly attempted to preserve. The result is a signal broad enough to implicate harmless and assistive use, yet fragile enough to be removed by a motivated person through substantial recomposition. It risks concentrating suspicion on ordinary and assistive users while remaining weakest against deliberate deception.
Without getting into the weeds, I still think Anthropic’s claim of their being no change in the meaning, quality, or readability is not accurate, using the plain meanings of those terms. If you read the papers, there are ways of ensuring that the frequencies of individual tokens or sequences of them are not affected in aggregate. But this is a different claim, and Anthropic does not specify which tradeoffs they’ve chosen there. I’m willing to grant that they can make the effect very small.
At this point, I think the most interesting question is, what will be the effects of these watermarks?
AI watermarking is security theater. There is absolutely nothing to prevent people from removing any possible watermark in text, images, video or other data. There is no way to prevent false positives, or false negatives, in detectors, either. It’s all superficial nonsense.
I find it interesting to be able to test a particular passage of text, but what am I supposed to do with that information? Even without watermark removal, a negative for a given model doesn’t tell you whether another model was used. And I’m uncomfortable with convicting someone based on a watermark given that false positives are possible and independent verification is not.
I asked Gemini about the benefits, and it said:
Regulatory Compliance and Policy
Legal adherence: Helps major tech platforms satisfy emerging mandates requiring automated systems to label machine-generated media.
Ecosystem tracking: Gives platforms and enterprises a high-level telemetry signal to monitor the spread of synthetic content across public channels.
Triage and Risk Mitigation
Mass moderation: Assists automated filters in flagging bulk, unvetted bot content or spam networks where statistical trends matter more than single-file perfection.
Provenance auditing: Provides an extra layer of metadata for digital media archives, helping investigators spot obvious clusters of AI-generated assets.
In other words, it’s not so much about individual pieces of content.
Also, more and more, I do think this has little to do with the EU regulation and everything to do with Anthropic and Google wanting to watermark the text that Claude and Gemini generate for their own purposes, to identify text to avoid training future models on to avoid model collapse/AI inbreeding.
Leading US AI labs such as OpenAI and Anthropic are releasing cheaper models as they fight to retain cost-conscious customers who are switching to cut-price alternatives from Chinese rivals.
The price war comes as rising AI bills push companies to curb usage and seek cheaper models, helping Chinese developers including Moonshot and DeepSeek make inroads with users from Silicon Valley to Europe.
OpenAI recently said that it was slashing prices for GPT-5.6 Luna, its “fastest and most affordable model”, by 80 percent. Anthropic has launched Claude Opus 5, touting the system’s “frontier intelligence... at half the price” of Fable 5, the company’s most capable model.
Anthropic has pledged to start marking Claude-generated text and images with machine-readable data, in an effort to comply with European rules for AI transparency. "Generated text will carry embedded watermarks, and generated files will include digitally signed provenance metadata where supported," Anthropic says on a new Claude support page. The changes are invisible to human eyes, but will make it easier for people and online platforms to detect if content was generated by Claude models.
Login
