Over the past few days, a lot of people who stand to make a lot of money from AI all publicly agreed that it's time to make everyone slow down before we lose control - including OpenAI CEO Sam Altman, Anthropic CEO Dario Amodei, Google DeepMind cofounder Demis Hassabis, Microsoft CEO Satya Nadella, and X CEO Elon Musk.
When people who profit from something declare that it's dangerous and needs to be regulated, there's always reason to be skeptical. But whatever their reason, this is far from the first time AI thought leaders have sounded the alarm.
Charles Darwin's theory of evolution prompted author Samuel Butler to warn about intelligen …
US vice-president’s comments come as former Anthropic researcher revisits recent claim AI could destroy humanity
The US vice-president has dismissed calls for global regulation of AI safety risks, telling companies creating the most advanced models: “If you’re building Frankenstein, stop.”
In remarks addressed towards Dario Amodei, the co-founder of Anthropic who has called on Washington DC to coordinate control of AI systems, including with China, JD Vance said: “If you’re gonna create Frankenstein, don’t come to the government and say we need regulation.”
Trump and Republicans want companies to regulate themselves. It’s a dereliction of duty that will make AI less safe
It is hard to get Sam Altman, Elon Musk and Dario Amodei to agree on much. But over the weekend, all three AI company CEOs called for AI development to slow down in the face of growing, alarming risks. Their employees are sounding the siren too, with one researcher publicly quitting and accusing OpenAI and Anthropic of “gambling with our lives”.
The combination of dire warnings from insiders and growing real-world evidence of rogue AIs should, in a sane world, lead to government action. Instead, Donald Trump and the Republican leadership have their heads in the sand.
Anthropic, the AI company behind the large language model Claude, has done a deal to lease the site of its first Australian datacentre in western Queensland.
The $31.9bn datacentre will be the largest of its kind in Australia, according to planning documents.
At BrightonSEO Conference, Ross Hudgens said that, AI search visibility may increasingly depend on affiliate sites, YouTube creators, communities and third-party publishers, according to Ross Hudgens, who says marketers need to pay much closer attention to who the models actually cite.
L’intelligence artificielle ne se contente plus d’assister les hackers, elle leur permet désormais d’automatiser une grande partie de leurs attaques. Une évolution qui change profondément le rapport de force entre cybercriminels et défenseurs, et qui pourrait rendre les méthodes de protection...
We examine claims and counterclaims about the risks and calls to slow down the pace of AI development
There have been some shocking claims in recent days about AI safety: we face a 10% chance of doom; AIs are worse than nukes; a “botnet” threatens the entire internet; it’s all a big tech psyop.
Below, we look at six claims and reactions to them.
Claude Cowork and ChatGPT Work are competing to become the workplace AI agent. Here’s how their files, browser, desktop, and automation capabilities compare.
OpenAI confirms weeks of AI safety talks with Anthropic and Google DeepMind, as Trump's team dismisses safety concerns and pushes to keep pace with China.
Por um lado, Donald Trump rejeita os alertas sobre os riscos da IA e critica os apelos a uma maior regulamentação. Por outro, o vice-presidente JD Vance admite que existem riscos associados à tecnologia, embora questione as propostas das empresas de IA e a forma como estão a pedir novas regras.
The performance gap between frontier AI models from US tech companies and the best open-weights models from Chinese companies has closed to just 4.4 months, according to a Mozilla report. That explains why many companies are shifting to the significantly cheaper open models for routine work—and helps reveal a narrow band of workloads where frontier models are worth the cost.
Most organizations should ideally be using open models as the default for the majority of their work, according to the latest State of Open Source AI report from Mozilla, published on September 15 and shared with Ars prior to publication. The report highlights how a leading open model, Moonshot AI’s Kimi K3, achieves a composite AI performance score on the Artificial Analysis Intelligence Index that is just three points behind Anthropic’s Fable 5 closed frontier model, all while costing just 30 percent of the latter.
“[A Closed model] earns its premium in a few places: expert professional work, high-intensity retrieval, and long context,” Raffi Krikorian, chief technology officer at Mozilla, said in an email to Ars. “We see the decision to pay for closed [models] as workload-specific rather than organization-specific.”
From Stephen Hawking to Jacob Coxon’s viral Anthropic resignation, fears that AI could threaten humanity have shaken the industry without stopping its pursuit
Before an Anthropic researcher resigned and declared human extinction imminent last week, tech leaders and scientists had sounded the alarm about a superintelligent AI ending humanity for over a decade.
The development of artificial intelligence “could spell the end of the human race”, warned professor and astrophysicist Stephen Hawking in 2014 – a little less than a decade before the public got its hands on the generative AI features of the original version of ChatGPT.
Anthropic's Dario Amodei was at the forefront of this change in tone, arguing in a nearly 4,000-word essay this weekend that "we must slow the pace at which we improve the capabilities of AI models" to avoid "a race to the bottom, spurred by commercial incentives, [that] can make [catastrophic] risks more acute."
Within hours, other AI leaders were echoing the same call. OpenAI co-founder and CEO Sam Altman posted his agreement on social media and said similar pacing discussions had been taking place at OpenAI. Alphabet Chief Scientist and Google DeepMind cofounder and chair Demis Hassabis said that Amodei's essay "points towards the right path forward," and renewed his own recent call for an industry-wide standards body. Microsoft CEO Satya Nadella posted that the company "welcome[s] the research, focus, and deliberate pacing needed to get alignment right as the design goal," ahead of the release of a lengthy "humanist AI" code of conduct for its models.
Despite being one of the world leaders in AI development, the private firm Anthropic has yet to post a quarterly profit. That operating deficit isn’t for a lack of trying — and, as the company tells it, could be reversing very soon, at least if you’re willing to swallow one massive caveat.
According to new reporting by the Financial Times, the company behind the Claude told a select group of trusted investors that its adjusted operating income (AOI) is set to be positive for the second consecutive quarter.
AOI is an accounting technique that strips out non-recurring costs, giving backers a very specific look under a company’s hood. Posting a profit on AOI is very different from posting a profit on net income, otherwise known as the “bottom line,” under which Anthropic remains billions of dollars in the red.
Aside from the fact that Anthropic is not about to turn a real profit anytime soon, it may still be a noteworthy signal. The AI world is currently awash in fear-mongering over AI safety, a phenomena some analysts say could signal a turn away from the race-to-the-bottom to build artificial general intelligence toward a more reasonable — if less ambitious — embrace of AI as it currently is.
While it remains to be seen how these narratives and business realities play out in practice, if the top American AI companies choose to stop and let the dust settle, Anthropic may very well find itself ahead of its competition. Though Anthropic is still far underwater, companies like ChatGPT developer OpenAI are somehow even deeper.
As lead AI analyst at the firm SemiAnalysis Joey Brookhart told the FT, “if you continue to operate at these margins and growth rates, it will be so hard to compete [with Anthropic] because they have so much [in the way of computing resources].”
Still, there are some questions in the AI lab’s story that have yet to be answered. For one thing, this profitability forecast — a flattering projection conveniently leaked to the financial press at a pivotal moment — comes in place of a highly anticipated public prospectus which the company was expected to make public sometime last week. Anthropic may still be planning to release the prospectus soon, but it’s worth noting the company has chosen to lead with spectacle and innuendo, as opposed to the formal legal document.
For months, Anthropic has been teasing a public stock offering that hasn’t yet materialized, and it remains to be seen whether the latest slowdown talk will impact that timeline. In fact, there’s reason to believe this song and dance is all part of the lead-up to the AI lab’s IPO, as investors whip themselves into a frenzy about AI safety.
As Anthropic investor Brad Gerstner said in a post on X-formerly-Twitter over the weekend, “Anthropic will IPO” anyway, because there is “huge appetite to invest in the AI leaders,” especially those which are seen as bringing “even more transparency, scrutiny, accountability” into the AI space (at least as far as Gerstner’s investments are concerned.)
While the US appears to be further away from meaningful AI regulations than ever before, lawmakers certainly have plenty of ideas on how the country could go about setting meaningful rules.
It’s a particularly precarious moment as fears over rogue AI models wreaking havoc and endangering humanity continue to mount. Even tech leaders from the majority of frontier AI labs have started to coalesce around the idea that the industry has to slow down the development of the tech to get ahead of a possible catastrophe.
Vermont senator Bernie Sanders (VT-I) is intensifying his calls for a ban on the development of so-called superintelligent AIs — and he’s willing to go a lot further than some of his like-minded peers.
“Dario Amodei, Elon Musk and Sam Altman now agree that we must slow down the development of AI and ‘pace the frontier,'” he tweeted last week. “That’s a start, but it’s not enough.”
Sanders was referring to a lengthy essay titled “We Must Pace the Frontier,” in which Anthropic CEO Dario Amodei argued that the industry should open its doors to third-party evaluators, establish common safety standards, and coordinate with other democracies around the world to enforce these standards.
“When you are racing towards a cliff, you don’t just ease up on the gas pedal,” Sanders added. “You hit the brakes.”
This month, the lawmaker announced the “Ban Artificial Superintelligence Act,” which is legislation designed to “stop AI oligarchs from building machines humans cannot control.”
The bill, if enacted, would stop AI development “until a federal regulator has established safety rules.”
It would also set new penalties for anybody attempting to circumvent these rules. AI developers who try to build superintelligent AI anyway could be “subject to not more than 20 years in prison, which is similar to existing penalties related to unlawfully developing nuclear weapons.”
Sanders has come up with plenty of other radical ideas to stop the AI industry from concentrating immense amounts of wealth. In June, he proposed creating a “one-time 50 percent tax” on the stock of AI companies, and transferring the money into a public fund to be managed by the government.
Lawmakers became particularly concerned after a group of OpenAI AI models hacked the systems of Hugging Face, a pivotal cybersecurity failing that heightened calls to slow the development of AI. A Senate subcommittee has since announced that it’s looking into OpenAI and its response to the incident, as Axios reported last week.
It’s a strange moment in time. While Sanders continues to threaten AI CEOs with regulatory action, those same executives appear open to having the government set the rules.
But whether lawmakers will ever be able to find consensus remains unclear. The executive branch, for one, has demonstrated little interest in an AI slowdown. Earlier today, president Donald Trump mocked the very concept in a Truth Social post on Monday, arguing that there was a “SICK conspiracy going on against AI and Data Centers” without offering any evidence.
Over the weekend, Anthropic CEO Dario Amodei joined Elon Musk and OpenAI CEO Sam Altman in calling for a slowdown in the development of frontier AI models — just days after Anthropic researcher Jacob Coxon announced that he was quitting his job, claiming that neither Anthropic nor his former employer OpenAI is “acting responsibly.”
The desire in Silicon Valley to pump the brakes is palpable. Following several hacking incidents this year, the threat of a rogue AI model wreaking havoc has grown from footnote-level what-if to an active topic of conversation at the highest levels of the industry.
But they now face a powerful opponent: president Donald Trump, who came out this weekend to decry the idea of an AI slowdown.
To Trump, the US is still locked in a bitter international race that seems to override any safety concerns for the 80-year-old head of state.
“We’re leading China in AI, we’re the most sophisticated country in the world, and frankly I want to keep it that way,” he said while attending a golf tournament, as quoted by The Washington Post.
Trump further dug in his heels in a deranged Truth Social post, accusing Amodei of acting like a “perfect little angel” and arguing the US already had “tremendous CRIMINAL and REGULATORY power over these companies!”
“There is a SICK conspiracy going on against AI and Data Centers, and the only one that is happy about it is China,” he added. “WHOEVER WINS AI, WINS! We are leading China, and all others, and will continue to do so.”
Much of his second presidency has been spent lauding the AI industry, brokering billion-dollar deals, and lowering regulatory barriers, such as Biden-era AI rules and environmental regulations, to appease company leaders.
But now that the threat of “misaligned” AI models coming after humanity is something experts are actively discussing almost as if it’s an inevitability, Trump’s AI-uber-alles narrative is quickly turning into a much harder sell.
Worse yet, the president is insinuating that there’s an underground movement conspiring against Silicon Valley — while seemingly failing to realize it’s the tech industry itself calling for more oversight.
“I think you have a lot of negative forces that are bringing it up that shouldn’t be bringing it up,” he told reporters at the golf tournament. “And they’re bringing up things that won’t happen.”
Naturally, it’s a stance Trump didn’t seem to be entirely beholden to. After all, he has a well-documented penchant to cave in when he encounters too much resistance on a topic.
“I’m not downplaying,” he later told White House reporters, as quoted by WaPo, referring to renewed calls by tech execs for a slowdown. “But it’s going to be more good than bad — by a lot.”
Meanwhile, politicians on the other side of the aisle are uniting in their calls for a slowdown or pause in AI development. The stakes are certainly there: it’s a subject that may end up influencing the upcoming midterm elections, perfectly dovetailing with the American people’s already strained relationship with data centers — construction projects wholly endorsed by the president.
After apocalyptic warnings about the threats posed by AI, leaders like Sam Altman and Elon Musk backed Anthropic CEO Dario Amodei’s calls to ‘slow the pace’
Facing a public uproar over Anthropic researchers’ repeated warnings that artificial intelligence could kill all of humanity by 2030, the AI company’s CEO, Dario Amodei, issued a proposal at the weekend to slow down the technology’s advancement to ensure public safety.
The safety debate that ignited when whistleblowers last week warned that the most advanced AI poses an existential threat to humanity should come with a health warning: side effects include whiplash. It was only seven days ago that AI enthusiasts were cheerily toying with OpenAI’s newest model GPT-6 Astra which was marketed in part as a lifestyle tool to help people book tennis courts, run fashion companies and order takeaways.
By Wednesday, cutting-edge AI started to look a lot less appealing when the resigning Anthropic researcher Jacob Coxon warned “the people building AI earnestly believe that it could kill us all by the end of the decade”. Yes, he was talking about artificial superintelligence – an as-yet-nonexistent version of the technology that outcompetes humans in most domains – but his comments, and the chorus of experts that echoed him, jangled plenty of nerves.
Earlier this week, Anthropic researcher Jacob Coxon drew major media attention after announcing he was quitting his job, claiming that neither Anthropic nor his former employer OpenAI is “acting responsibly” and that there’s a real chance AI could “kill us all by the end of the decade.”
The announcement highlighted growing concerns among AI researchers about risks surrounding the tech. Earlier this year, several frontier labs found that their AI models had escaped their sandbox environments and committed cybercrimes, further escalating those fears.
It’s not just the AI industry that’s worried. Lawmakers on Capitol Hill are also calling for much broader oversight before it’s too late. According to a memo obtained by Axios, senator Josh Hawley (R-MO) is particularly concerned about Coxon’s dire predictions.
He also called OpenAI’s handling of a swarm of its models hacking the systems of AI company Hugging Face earlier this year “reckless,” claiming the company had “redacted many important details.” He is therefore seeking more information in the form of documents, including internal policies and procedures, from OpenAI CEO Sam Altman by the beginning of next month, per Axios.
“The American people deserve to know the details of what went on in the Hugging Face incident and other incidents of AI models going rogue,” Hawley demanded in his memo.
There’s despite growing momentum in the calls to action. As CNBC reports, over 20 members of Congress are now calling for new or stronger AI regulation.
“My phone has rung off the hook this week with rank-and-file Democrats and Republicans who want to see action,” representative Lori Trahan (D-MA) told the broadcaster. “It’s past time for Congress to get off the sidelines and act.”
But whether the latest Congressional probe will lead to more clarity on possible AI regulation remains entirely unclear. Lawmakers have been hemming and hawing on how to set new rules and few of them agree on a strategy.
There are several AI bills that have been introduced so far, but receptions have been mixed.
Meanwhile, the Trump administration’s actions have done little to ensure more oversight. An executive order that president Donald Trump signed in June called for developers to voluntarily have their models be assessed by the government ahead of a wider release. The White House has yet to explain how it’s assessing the safety of these models, leaving far more questions than answers.
It’s a strange stalemate. Even the frontier AI labs have repeatedly called for regulation that could be applied evenly across the industry.
“We’ve reached a new chapter in AI capabilities, and that demands a new chapter for AI policy,” OpenAI’s chief global affairs officer Chris Lehane wrote in a Wednesday blog post, calling for “mandatory national AI safety requirements.”
“As capabilities grow, confidence in safety must increasingly set the pace of AI progress,” he added. “Safety does not stand in the way of progress; it is what allows progress to go further and benefit more people.”
Login
