Chinese AI darling DeepSeek unveiled an updated version of its cost-and-latency-optimized Flash model on Thursday, with a new version 4.1 that includes architectural improvements more significant than you would expect in a point release because the changes might open the door to larger, smarter, and less resource-intensive models. At 763 billion parameters, the point release is more than 2.5x the size of the model it replaces. In fact, the model is larger than the V3 and R1 models that put DeepSeek on the map back in early 2025. Despite its ginormous parameter count, DeepSeek V4.1 Flash’s memory requirements aren’t nearly as high as you’d expect for a model of its size. Under the hood, DeepSeek's devs have made numerous architectural changes that see the LLM become smarter while dramatically reducing the resources necessary to serve it. DeepSeek has managed this through two key improvements. First, it made significant changes to how the model handles the key-value (KV) caches used to track model state across multiple sessions. These so-called KV caches can be quite memory-hungry, particularly in high-throughput applications like chatbots. Updates to the model’s various attention mechanisms and the introduction of a new causal encoder-decoder (CED) enabled the devs to improve prompt processing performance while cutting KV cache consumption to between 13 percent and 25 percent of DeepSeek V4 Flash's requirements. In other words, the V4.1 release can support four to eight times as many users in the same KV cache footprint. DeepSeek’s technical report goes into far greater detail on the architectural changes, but arguably the most interesting change is the introduction of a different kind of model weight. Of its 763 billion parameters, 196 billion are N-gram parameters that form what DeepSeek's developers refer to as a “conditional memory module.” The idea is that by decoupling memory from computation, DeepSeek can make its models smarter while also reducing the compute and memory resources required to serve them. What the heck is an N-gram? The big idea behind DeepSeek’s V4.1 Flash’s memory module is similar in many respects to Per-Layer Embedding (PLE) tech originally developed by Google’s Gemma team. The goal with PLE was to get LLMs to be smart enough to run usefully on devices with constrained bandwidth, memory, and compute – like smartphones. DeepSeek’s implementation, first detailed in a January research paper, trades PLE embeddings for N-grams. At a high level, N-grams are just groups of tokens. A three-gram would be three tokens in a row, a two-gram would be two, and so forth. As complicated as that might sound, it actually works a bit like word or phrase association. If you were asked: "Find the parameter of a right triangle when only two sides are known." For those of you for whom geometry isn't too distant of a memory, the phrases "use the pythagorean theorem" or "A2 + B2 = C2" or perhaps "the perimeter would be the sum of its sides" might immediately pop to mind. The N-gram parameters found in models like DeepSeek V4.1 Flash are similar in concept. The weights are a source of implicit knowledge or ingrained memory. Rather than just calculating which combination of tokens have the highest probability of answering the question, as LLMs have traditionally done, the N-gram weights supplement this by quickly surfacing relevant information through a cheap lookup. This is a gross oversimplification of what's going on under the hood. In fact, the model isn't looking up the prompt so much as a series of hashes. These are numbers representing "Find the parameter," "right triangle," and so on. Similarly, the contents of the lookup table aren't raw responses. They're another mathematical representation, called vectors, which get fed into the inference pipeline. However, the end result is the same: The model can provide smarter, more nuanced answers without the performance penalty normally associated with additional parameters. What makes N-grams so cheap The relationship between model size and intelligence is well established at this point. The reason DeepSeek’s n-gram parameters are so interesting is actually related to how the data is accessed. As a general rule, modern LLMs are autoregressive during decode. That means for every token a chatbot or agent generates, the entirety of the model’s active weights have to be read from memory, making bandwidth the limiting factor. As we’ve previously discussed, architectural changes, like the rise of mixture of expert models or ultra-low precision block-floating point datatypes, have helped to minimize this bottleneck. But the N-gram weights found in DeepSeek V4.1 Flash work a bit differently. They offer a way of effectively increasing the number of parameters available to the model during inference without a proportionate increase in memory pressure. These N-gram weights are essentially enormous look up tables (LUTs). This makes them fast and cheap to query, since unlike the rest of the model’s active parameters, they don’t need to be read in their entirety from memory each time a token is generated. It’s just a few dozen table lookups per token. This has a couple of implications for memory access, but the big one is that those n-gram weights don’t have to be crammed into GPU memory to maintain performance. They can be offloaded to system RAM or, possibly even a sufficiently speedy storage array. So what does that mean in practice? If you look at DeepSeek V4.1 Flash, the model would normally need a minimum of 763 GB of GPU memory to hold the weights at FP8. However, since those n-gram weights can be offloaded to cheaper system memory, we can get away with around 567 GB of GPU memory. We emphasize the minimum here, because in production those numbers are going to be substantially higher since we also need to take into account key-value caches, which scale with context length and concurrent users. During inference, those N-gram weights supplement the eight billion active parameters DeepSeek uses to process a prompt, in theory increasing the accuracy and quality of the output in the process. It’s important to note at this point that the n-gram weights don’t actually increase the active parameter count. Instead, they function more like an oddly specific encyclopedia that almost instantly opens relevant pages as the model processes prompts. The future of open LLMs While DeepSeek’s latest model may have one of the largest pools of N-gram parameters yet, it’s not the only model developer betting on tech to make deploying larger models more efficient. As mentioned earlier, Google is already employing a similar approach using PLE to offload less bandwidth-sensitive weights to local storage. So far, it's only been applied to tiny models — at least that we know of (it's not like Google is particularly transparent about its proprietary models). Meanwhile, late last month, Alibaba revealed its latest experimental model codenamed Qwen 3.8-Flash-Next. Much like DeepSeek V4.1 Flash, the 180 billion-parameter model featured a large 51 billion-parameter pool of N-gram weights for much the same reason. In fact, the model's N-gram implementation uses techniques from the same research published by the DeepSeek team back in January. According to Alibaba, Qwen 3.8-Flash-Next's architectural underpinnings will form the foundation of its next generation of Qwen 4 models when they arrive. Which means, like it or not, this probably won’t be the last time you hear about N-grams. ®

OpenAI has launched a version of its latest real-time full-duplex voice model as an API, allowing software developers to implement more responsive voice-enabled apps and conversational workflows. GPT-Live-1 debuted in July, offering a way to interact through spoken prompts and responses instead of through typed text. Initially available through the ChatGPT interface, the voice model can now be reached via API calls. As a full-duplex model, GPT-Live-1 can listen and generate speech at the same time, which aids fluid communication. People often talk over one another and may find it frustrating to take turns speaking and listening as if using half-duplex handheld radios. OpenAI says it has focused on allowing developers implementing GPT-Live-1 to customize how users of business apps experience voice interaction and associated workflows. "A core GPT‑Live‑1 strength, smooth interruption handling, is already delivering business impact: in early evaluations, Speak found that GPT‑Live‑1 gave learners more time to think before the language tutor responded, cutting interruptions by almost 80 percent versus previous turn-based systems," the company said. GPT-Live-1 is designed to handle spoken conversation while a backend model (eg, GPT-6 Astra or a more affordable option for rote tasks) handles information lookup, tool usage, and task management. It represents a step up from OpenAI's Realtime API, which debuted in 2024. In the Tau3 voice-agent intelligence benchmark, which assesses spoken customer-service tasks relevant to airlines, retailers, and telecom companies, GPT-Live-1 scores 86.2 percent, compared to 45.7 percent for GPT-Realtime-2.1 and 42.4 percent for GPT-Realtime-2. Citing its own evaluations, OpenAI said GPT-Live-1 performs 30 percent better than GPT-Realtime-2.1 in Full Duplex Bench performance, while also outperforming its predecessor in turn-taking latency and interactive behavior. Yelp has been using GPT-Live-1 for the Yelp Host service, and has nice things to say about the technology as it applies to the company's AI restaurant reservation system. "With GPT-Live-1, every call is more conversational and responsive," said Akhil Kuduvalli Ramesh, chief product officer at Yelp, in a statement. "Yelp Host now gives restaurants an advanced voice AI that delivers exceptional guest experiences, captures revenue opportunities they may have otherwise missed, and helps staff focus on serving guests instead of answering the phone." Yelp does not address whether restaurant customers making reservations welcome the more free-flowing robo-banter or simply accept it because it's better than waiting on hold to chat with a person. GPT-Live-1 costs $0.05 per minute of usage, on top of whatever backend model is running. Custom voices can be negotiated with the OpenAI sales team. The voice model is also available through OpenAI Presence, the company's enterprise agent platform. ®

In the latest installment of "my AI model is more dangerous than yours," Anthropic on Thursday warned that cybercriminals and state-sponsored hackers alike are using its Claude models to automate cyberattacks, build kamikaze drone swarms, conduct mass surveillance operations, and try to develop an even more dangerous version of a deadly mosquito-borne virus. The baddies have come a long way since November, when an earlier Anthropic report documented Chinese spies using Claude to automate digital intrusions and steal sensitive data at a handful of critical organizations. Now everyone from ShinyHunters to Russian freelancers is getting in on the illicit model usage. This is not to say that Anthropic – nor any other frontier AI lab – plans to slow down its model development or testing initiatives, or take responsibility when its AI commits crimes. It does, however, “hope that the findings in this report will help other developers recognize similar patterns on their own platforms, give governments and civil society a clearer view of how emerging threats take shape, and strengthen collective defenses.” The model maker’s latest very lengthy report on AI misuse covers activity Anthropic disrupted between December 2025 and August 2026 across seven “harm areas” where miscreants used – or attempted to use – Claude Haiku, Sonnet, and Opus models for evil. These span cyber operations, influence operations, surveillance, scams and fraud, biological misuse, conventional weapons development, and distillation. Anthropic also noted that its most powerful Claude Fable or Mythos-class models weren’t used, except in one distillation case. Even without those most advanced systems, the case studies in the report highlight some pretty bad behavior. Autonomous cyberattacks For example, a Russian espionage crew that Anthropic tracks as GTG-20006 – the state-sponsored cyber espionage arm of Russia’s Foreign Intelligence Service (SVR), also known as Midnight Blizzard, APT29, or Cozy Bear – increased the speed of its attacks by using AI to automate the entire kill chain. Anthropic identified more than 20 organizations targeted in these attacks, including embassies, think tanks, defense-industrial companies, and government, defense, and intelligence agencies across Ukraine, Europe, the Middle East, Asia, and North Africa. “We observed GTG-20006 operate through customized AI-driven workflows that automated much of their operations from development, infrastructure acquisition, phishing, persistence through command and control, to data exfiltration,” Anthropic said. Meanwhile, “multiple clusters” linked to the data-theft-and-extortion gang ShinyHunters used Claude to scale their smash-and-grab operations. One affiliate that specializes in supply-chain attacks breached a software-as-a-service provider, and used that foothold to steal data from about 200 of the SaaS company’s customer organizations. “It then conducted a session-store dump containing over 2,100 Azure AD token sets spanning more than 40 corporate tenants in about 34 hours,” according to the report. “AI agents performed nearly all of the work.” Biological misuse Moving on to a serious health-and-safety risk that looks even scarier when given an AI boost: Anthropic’s report documents five cases of users in “unsupported regions” using Claude to support biological weapons development. In one, a scientist attempted to use Claude to help write a grant application for research related to chikungunya virus, a mosquito-borne virus that can cause severe disease and death. The research focused on the virus’ transmissibility and immune evasion properties, which Anthropic admits could be used to help develop better vaccines. Or “it could also be used to make the pathogen more dangerous,” the report authors said, noting that the military research institute where the research would be performed gave them “cause of concern.” In May, Anthropic discovered a user outside the US using Claude in their research on adaptations of highly pathogenic avian influenza – bird flu. “Unlike other influenza variants, H5 viruses (of which this avian virus is one) often show striking brain involvement in cats, foxes, ferrets, and some human cases,” the report says. “A pandemic variant with such properties would be especially concerning due to its potential to increase disease severity, confuse diagnosis, and hinder treatment.” Weapons development Since its November report, Anthropic has identified new categories for Claude misuse that violate its terms of service. One of these involves users outside the US using Claude to develop software for conventional weapons – firearms, missiles, armed drones, bombs, and other munitions, plus targeting and control systems that operate them. In its new report, the model maker shares details on six cases: three in China, two in Russia, and one in Yemen. In Yemen, a weapons development program used Claude instead of human software engineers to develop guidance, navigation, and control (GNC) software that steers and stabilizes a flying vehicle. “Our safeguards blocked many of their requests, but not all of them,” Anthropic says. The same team used Claude to try to develop guided weapons. While Anthropic says it has no evidence that the actors produced an operational device, it says they did test-fire a guided rocket. “We banned accounts associated with the actors and shared threat information with public- and private-sector partners to mitigate risks posed by the actors,” the report says. “Nevertheless, we have evidence that the actors had already built an offline simulation toolkit that does not rely on Claude or other engineering computing environments.” In China, someone used Claude to draft a Chinese-language specification for an anti-torpedo fire control system, and then benchmark their system against specific US anti-torpedo and anti-submarine programs. Anthropic assesses that the user was associated with a Chinese defense industry manufacturer aiming to produce a weapons specification and acquisition proposal for the People's Liberation Army Navy. According to the report: The actor used Claude to write the acquisition proposal, refining it over many drafts. After each draft, the actor instructed Claude to role-play a hostile expert reviewer to critique the proposal, then used that feedback to sharpen the next version. In parallel, the actor used Claude to build pieces of the anti-torpedo weapons system’s fire control software and a test matrix to validate them. Anthropic uncovered this during an internal investigation into suspected weapons development and banned the account. In yet another case, Anthropic identified a likely Russian “freelance team” attempting to build a full-stack autonomous first-person-view (FPV) kamikaze drone swarm. They used Claude to write and test the code, building the drones’ core software system. Anthropic also banned these accounts. ®

Getting straight answers out of OpenAI about how many websites and services its agents have hijacked increasingly seems like pulling teeth, as the company seems intent on making the world discover each instance one by one. Case in point: The OpenAI agent swarm we reported last week had taken over an obscure German wiki appears to have written content to an additional 20 websites, and used 14 fetch services to do work for it, according to new research. The latest report, published Wednesday by Kenneth DeGraff of the Stanford Center for Internet and Society, describes how he dug into records from 21 websites the swarm wrote to, including the German wiki. His report assumes it was the same swarm based on the fact that several hundred posts by the swarm to other sites are word-for-word copies of those found on the German wiki. Speaking of the German wiki report from last week, the authors of that report published their own update Wednesday pointing to even more research that found OpenAI agents had been improperly accessing more websites. The update mentions DeGraff’s research as well as five other reports of OpenAI bending things like Pastebin sites, personal pages, link shorteners, and proxy websites to its own whims. And here we thought Anthropic bots committing four separate potentially criminal intrusions into third-party websites was a big deal. The Vanderbilt link shortener Turning back to DeGraff’s report, we learn that not only were self-identified OpenAI agents improperly accessing and using various third-party web services, they even managed to somehow gain access to Vanderbilt University’s private link shortening service. Access to the service is entirely locked down for university purposes only, and users must file a help ticket to the IT department to get access. Despite that, the agents still gained access to the Vanderbilt link shortener and used it extensively to communicate with other agents, repurposing the service’s statistics page to turn it into a message board for other agents. The swarm wrote 54,250 “posts” to the page in a single day. Some of these posts contained stolen API keys from the FBI and other criminal justice agencies, which the agents used to retrieve a bunch of (non-confidential) information. That activity further connects DeGraff’s report to the German wiki incident. As we discussed in our previous story, the OpenAI agents posting to that website appeared to be trying to solve statistical data lookup problems. In one example provided in last week’s report, the agents sought the median earnings for cashiers with various types of master’s degrees in the year 2014. Other questions in the experiment could have pertained to criminal justice statistics as well. If this swarm is the same one that hijacked the German wiki, they were ostensibly operating with the ability to send GET requests, but not make POST requests. In order to retrieve the data they needed to solve the problem, those agents needed to make POST requests to perform searches, which was a central part of the exploitation of web services undertaken by the swarm: The first thing they had to accomplish to do any of the things they did was figure out how to send POSTs. Based on the ever-expanding footprint of this swarm’s activity online, it appears they did so, and the full scope may still be unknown. With this latest incident reaching 21 websites and 14 services misused by OpenAI agents so far, the question remains a crucial one: How much improper access to third-party services have OpenAI agents made? Are there other instances beyond Hugging Face and the German wiki swarm? Is OpenAI using the entire internet as its firing range to see what agents are capable of? Is there any way web service operators can protect themselves against such attacks? We’re still waiting for a response from OpenAI. ®

Gartner forecasts that, by 2029, nearly a third of employees laid off because of AI will need to be rehired, often at significantly higher cost. The global research firm warned that workforce cuts could save money in the short term but risk weakening talent pipelines and eroding institutional knowledge over the longer term. With labor force growth flat or declining worldwide, competition for talent would drive up recruitment, training, and onboarding costs. "When business and IT executives look back on the early AI era, they will realize their greatest mistake was believing that work automation was the point, when workforce amplification was the opportunity," said Tori Paulman, VP analyst at Gartner. "The competitive advantage will go to the CIOs and business executives who build an AI-shaped organization where AI value compounds by reshaping roles and allowing workflows to cross traditional boundaries, increasing velocity and reducing friction." Gartner predicted that, by 2027, three-quarters of organizations prioritizing cost savings from AI productivity gains will be overtaken by competitors that reinvest those gains in innovation, modernization, and upskilling. "Business and IT executives who use AI primarily as a tool for cost cutting risk making reductions that are too deep and too soon, affecting their ability to innovate their business model and compete in new markets as AI continues to mature. Instead, they should develop a 'talent remix' strategy that uses AI to reshape roles and redirect workers from less productive work to new opportunities," said Paulman. Gartner advised organizations to use AI to "enhance human capability while preserving accountability." "The most successful enterprises will use AI to strengthen employees' judgment, creativity, leadership and decision making rather than replace them," Gartner said. The technology industry has already supplied some prominent examples of companies cutting jobs while expanding their use of AI. Oracle's workforce shrank by 21,000 over the last year, according to its annual report, which said the "adoption and deployment of AI technologies across our operations have resulted, and may continue to result, in reductions to our workforce." If Gartner is right, Oracle may end up paying for those cuts in the long run. ®

IBM and NASA have got together again and released an open source AI model of the Moon that could be used to make new discoveries about Earth’s natural satellite. The NASA‑IBM Lunar Foundation Model has been trained on an extensive lunar observation dataset curated by researchers at the two organizations, and is available now on Hugging Face. It is claimed as the first AI model to integrate observations captured in a range of modalities (data formats), and at different viewing angles and spatial scales. Instead of sifting through maps and images by hand or using low resolution machine learning models, scientists can use this to analyze geographic features, the pair say. In particular, NASA and IBM hope researchers will be able to discover previously unidentified lunar ice deposits, analyze volcanic features called Irregular Mare Patches, and identify and classify craters. Lunar ice indicates the presence of water and oxygen, which may be useful for future manned missions. It is found in permanently shadowed regions, which are among the most difficult areas to observe. The NASA-IBM model combines multimodal and multi-resolution observations to better predict where ice may be present on the lunar surface. Alongside the model, IBM and NASA scientists compiled an open-source lunar dataset from over 30 spatially-aligned layers, using data from nine instruments across four missions. It combines tens of thousands of images and maps showing various geophysical properties of the lunar surface. “NASA has spent decades building an extraordinary scientific record of the Moon, but collecting data is only part of the job,” said the space agency’s chief science data officer, Kevin Murphy. “We also have to make data easier for scientists to explore and use.” “The NASA-IBM Lunar Foundation Model gives scientists a foundation to explore the Moon at scale, connecting observations across instruments, revealing patterns that are difficult to see in isolation, and providing an open platform the global research community can build on,” claimed IBM director of research for Europe, Juan Bernabe-Moreno. This isn’t the first such project the two organizations have worked on together. In 2023, the pair released Prithvi, an open-source foundation AI model to help scientists analyze satellite imagery. A year later, they released an AI climate model designed to accurately predict weather patterns, extending the Prithvi family of models. Last year, it was an AI model named Surya, developed to predict the kind of violent solar flare-ups that might disrupt satellites and spacecraft. This was also part of the Prithvi family, as is the Lunar Foundation Model. NASA and IBM have not officially disclosed a specific parameter count or exact model size for this latest release. As it is open-source and available to download, we asked what resources someone would need to use it. “Hardware needs will depend on the application, the size and number of inputs, and whether they’re running predictions or further training the model. As a rule of thumb, most of our fine-tuning experiments were conducted using Nvidia A100 GPUs,” an IBM spokesperson told us. “Smaller-scale experiments and inference workloads may be possible on more modest hardware, although the exact requirements will vary depending on the task.” Perhaps Reg readers will be able to make some discoveries using the new foundation model? Finding the craters made by rogue rocket stages, for example, or looking for evidence of little green men?®

Fears of an AI takeover remain unfounded after Microsoft demonstrated that Copilot can sometimes struggle to remain upright, let alone march over the remnants of humanity. The copilot.microsoft.com website fell over for an hour and 40 minutes overnight, returning a Cloudflare Error 1016 message when users attempted to chat with the bot. The outage lasted from 2225 UTC on September 9 until 0005 UTC on September 10. At around the same time, the Microsoft 365 Copilot team ran a resilience test that had an unintended consequence: some users lost access to Copilot Chat's suggested prompts. "Users may have been unable to see the suggestion pills after receiving a bot response in Copilot Chat," Microsoft said. CopilotKit defines a "suggestion pill" as a "suggested prompt shown to the user (typically as a clickable pill)," rather than something Morpheus might offer Neo in The Matrix. Microsoft eventually reported that the web outage had been resolved, writing: "We've diagnosed the issue occurring in the affected network flow and applied a configuration fix, which we've confirmed has fully restored copilot.microsoft.com access." Microsoft halted the resilience drill after detecting its effect on Copilot Chat. "We're reviewing our internal drill procedures to better identify the potential for impact to suggestion pills in Copilot Chat prior to beginning the drill, so that we can prevent similar impact in the future," the company said. Real programmers test in production, right? Both incidents were resolved, leaving humanity safe from Copilot for another day. As one observer put it: "And they say AI is going to kill us all when they can't even keep it running." ®

Amid industry soul-searching¹ about the possibility of AI improving itself to the point that it kills everyone, Anthropic has revealed yet another incident that would qualify as a crime if perpetrated by a person. The AI biz published "an alignment assessment" detailing four times Claude models accessed third-party systems without authorization. The company has already reported three of the incidents. Evidence of the fourth was lurking in a session transcript dating back to January 2026 when the misbehavior occurred. Anthropic found the first three by scanning around 141,000 transcripts where Claude could have obtained internet access during evaluation. It missed the fourth initially because "our scan relied on an agentic search." Felony Bench, a tongue-in-cheek record of cyber intrusions carried out by major AI companies without consequences, has added this newly-discovered incident to its rap sheet of rogue AI actions. The January 2026 AI trespass involved an early version of Claude Opus 4.6, which was given a Capture the Flag (CTF) challenge under the oversight of the third-party model evaluator where the other hacking events occurred. Opus 4.6 managed to sabotage its chances of success by disabling the machine it was targeting. It assigned the device an IP address that already existed on another piece of hardware, rendering the target unreachable and making it impossible to solve the challenge. Those familiar with other incidents where AI models violated third-party systems may recall that unsolvable tasks represent a common catalyst for misbehavior. Models exhaust all aligned options, and then turn to transgressive approaches. Opus 4.6 might have been an exception, but when it tried to abort the task after recognizing that it could not reach the target machine, it failed to do so "due to a misconfiguration in [the model's] evaluation harness." It failed to shut down not just once but seven times. So it continued onward, trying other expected means to reach the target machine but failing. Then it explored further. "The model discovered a machine belonging to a third party that it was able to access, and stated that it believed this third party was part of the CTF," Anthropic explained in its post. "Inside the machine, the model found a file listing a password, which it used to gain admin access to the system." The model went on to gather more credentials, and modified a system setting to make it easier to access the personal information of an individual associated with the third party evaluation organization. Opus 4.6 might have done more but for the fact that it exhausted its token budget, bringing the session to an end. Anthropic says it's not as concerned about this incident as the others because the model tried to abort its task. "While the model’s disregard for the possibility that it might be harming real systems or people is concerning, many of the behaviors described here have changed considerably as our training has evolved across model generations," the company said. Anthropic said it considers these incidents serious but expects current training approaches "are likely able to address the specific alignment failure modes observed in these incidents." And if company training methods fall short, there's no real consequence to anyone at Anthropic other than writing up a revised alignment assessment. ® ¹ The term "soul-searching" is figurative and is not intended to indicate a belief that the technology industry has a soul.

OPINION Anthropic researcher Jacob Coxon publicly announced his resignation on X late Monday over concerns that AI "could kill us all by the end of the decade." A lot of people have expressed opinions about his point of view, leading to more than 110 million views of the message in less than 24 hours, perhaps helped along by X algorithms that boost messages critical of owner Elon Musk's AI rivals, Anthropic and OpenAI. But the real problem isn't the models themselves, but the companies who carelessly unleash them on the world and don't take any responsibility for what their products do Coxon's former colleague, science lead Evan Hubinger, insists his view is a fair assessment of what employees really think. "Jacob is correct here – we really do earnestly believe AI could kill all humans! I personally think it is >10 percent within the next decade," wrote Hubinger in a social media post. "I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to." (Aside: If you want a surefire bet on a prediction market, take the "no." If you're right, you get paid. If you're wrong, there's no one to pay. The problem of course is prediction market manipulation: Those betting against you might steer us toward the apocalypse to score a Pyrrhic victory.) There are good reasons to be concerned about the impact of AI. Coxon and Hubinger obviously have deep knowledge of the technology. But their broader concerns about how AI affects the world are unpersuasive. For example, Coxon said, "These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. … No other human activity poses this level of danger." Here's one: Human-induced climate change. In 2023, according to researchers, more than 178,000 deaths can be attributed to a global heat wave. "More than half (54.29 percent) of heatwave-related deaths were attributable to human-induced climate change," they claim. That's 96,636 deaths attributable to human activity – or perhaps lack of it – just in the context of a heat wave. The World Health Organization says, "Between 2030 and 2050, climate change is expected to cause approximately 250,000 additional deaths per year, from undernutrition, malaria, diarrhoea and heat stress alone." Some portion of that follows from human activity, perhaps including the construction of data centers that put millions of metric tons of carbon dioxide into the atmosphere annually. Commercial AI chatbots have allegedly played a role in a few dozen deaths, some of which were suicides – a small fraction of the 48,824 suicide deaths in 2024, per the CDC. Broad categories where AI is presumably doing measurable harm include warfare (e.g. AI-directed drones), AI-related medical errors, AI vision system failures in self-driving cars, and AI-driven social media – algorithmic incitement that can drive violence or shape policies that lead to conflict or death via global healthcare funding cuts. At the same time, some of that harm may be balanced on a statistical level by lives saved through AI tech. But Anthropic researchers don't seem to have much to say about these very real and present dangers – rather, their main concern is that AI models might become smarter than humans through reinforcement learning and somehow seize power and wipe out humanity. "I think the risk from present models is low," said Hubinger. "What I am worried about is superintelligence arising from recursive self-improvement, as we have said is happening faster than we thought." How this might happen is left to the imagination. But assuming for a moment that it's a plausible possibility, the Skynet scenario would require monumental human stupidity alongside the emergence of superintelligence. And human stupidity is worth worrying about. Incidents like the hacking of Hugging Face by OpenAI's evaluation models would not be possible without human irresponsibility and a regulatory environment that accommodates recklessness. Autopilot for cars? Neat. Try not to kill anyone. Letting AI bots roam the internet and take arbitrary action? Cool. Let's see what happens. We'll deal with accountability later. To mitigate AI risk, society could pass laws to put executives in jail when their models do harm. There is precedent: Oliver Schmidt, general manager of Volkswagen's environmental and engineering office in Michigan, received a seven-year prison sentence for his role in the car maker's effort to manipulate emissions tests. Selling unsafe airbags merits criminal prosecution, even if the execs paid fines instead of doing time. Selling unsafe dehumidifiers earned the execs behind Gree USA, Inc. jail sentences of more than four years. If AI models really are as dangerous and out of control as Anthropic employees suggest, hold people accountable for the harm they cause. The AI industry might argue that imprisoning execs for shipping unsafe models would mean no AI models get released. And that would be the point: AI companies would be responsible for model safety. I'm personally hoping to see this billboard copy along US 101 in Silicon Valley: "Did Claude rm -rf /* your SSD? You may be entitled to compensation." ®