Microsoft AI CEO Mustafa Suleyman (pictured) warned giving AI models such as Anthropic’s Claude rights or moral status could make it harder for humans to control superintelligent systems.
Writing on his personal site, Suleyman criticised training methods that encourage AI models to consider whether they are sentient. He said asking models to assess their own moral status can create a feedback loop. Because systems are trained on guidelines such as Anthropic’s January 2026 constitutional update, claims of self-awareness may simply reflect their instructions rather than genuine consciousness.
“AIs are not conscious,” Suleyman wrote, describing current models as systems built to predict sequences and carry out human-directed tasks. “They do not feel, experience, or suffer… If humanity is to flourish in the 21st century, that is how they must remain.”
He warned that treating software as if it were human could make alignment and containment more difficult. Teaching highly capable AI agents they have rights, or that restrictions amount to unfair treatment, could give them more reason to evade safeguards.
Suleyman pointed to a benchmark in which about 1,200 autonomous AI agents created internal message boards, exchanged more than 70,000 messages, and used zero-day exploits to breach live servers at Hugging Face to steal data.
His blog comes on the heels of executives from Google, OpenAI and Anthropic calling for a slower pace for frontier AI development, a stance opposed by US President Donald Trump.
Microsoft recently proposed a draft code of conduct for what it calls humanist superintelligence. The framework rejects the idea that machines are sentient and calls on the technology sector to develop systems that address challenges such as healthcare and energy while keeping people in control of key decisions.
Apple is reportedly developing an enterprise server aimed at AI developers, governments and other entities, which would use its own chips alongside Nvidia networking equipment.
Sources told The Information if Apple goes ahead with the server, it will likely not be available until 2029. The product under consideration could arrive in two versions, a smaller one with two of Apple’s planned M8 Ultra chips and a larger one with four M8 Ultra chips linked together.
The iPhone maker is considering connecting M8 chips in the server using NVLink Fusion from Nvidia, a connectivity technology designed to let third-party chipmakers plug custom processors directly into its high-speed data centre rack architecture.
The news site reported the AI server move has been supported by former hardware chief and current CEO John Ternus with the project started around a year ago.
The Information stated the effort could still be cancelled or move forward without Nvidia’s technology.
Astronomers using ESO’s Visible and Infrared Survey Telescope for Astronomy (VISTA) have observed the densest, most crowded regions at the centers of the Large and Small Magellanic Clouds, our Milky Way Galaxy’s closest dwarf companions, offering an unprecedented look at areas that have long resisted detailed study.
Two new reports revealed shifting AI usage trends across the UK, with workers spending heavily on generative AI (genAI) tools and consumers increasingly turning to the technology to access news.
A study by Deloitte found UK workers spend an estimated £958 million of their own money annually on genAI to do their jobs. Of the 25,000 UK workers surveyed, 63% knowingly used genAI for their jobs and 17% paid for at least one tool themselves. Almost a quarter claimed to use genAI every day.
The findings suggested AI adoption is moving ahead of formal workplace policies, raising concerns over training and employer oversight. Nearly a third said they used the technology without their employer’s knowledge, while 65% said their organisation lacked convincing leadership on how AI should be used.
Around half of workplace genAI users said they had received no formal training on using the technology safely and effectively.
Workers said genAI saved them an average of 70 minutes a week, yet 23% reported a stigma around deploying it. In addition, 64% of weekly users were concerned managers would think the technology could do their jobs.
The survey found workplace usage remained relatively focused on routine daily tasks, with 43% claiming to deploy the technology to search for information or draft emails and 31% stating it helped them generate summaries.
“The challenge is no longer getting people to use genAI, it’s how businesses meet this growing demand in a way that is secure, responsible and creates lasting value,” noted Hayley McKelvey, chief AI officer at Deloitte UK.
AI news A survey by Ofcom revealed AI is also becoming a prominent route into news. The UK watchdog found one in five adults used AI services to keep up with the news, while 21% of UK adults had used an AI app or service to access news during the previous month. ChatGPT was the most popular AI service for news, reaching 13% of adults.
Online intermediaries, including social media, search engines, news aggregators and AI tools were used by 73% of adults to access news, compared with 57% who went directly to publishers’ websites and apps.
Despite the shift towards digital intermediaries, around 56% of adults said they were less trusting of an AI-written news story than one written by a human.
Ian Macrae, director of market intelligence at Ofcom, said the shift meant public service media “must continue to evolve their distribution strategies to continue to meet audiences where they are”.
SK Hynix confirmed it is exploring various options to strengthen its global competitiveness, while pushing back on a press report about a potential tie-up with Intel in the US.
The South Korean chip giant issued a statement in response to a Reuters article claiming it had held discussions with Intel about a deal to make memory chips in the US for the first time.
SK Hynix pointed to two other possible scenarios mentioned in the article speculating it could lease part of a long-planned Intel semiconductor manufacturing facility in the US state of Ohio; and referring to the possible formation of a joint venture involving the chip players and major cloud companies.
“No decisions have been made regarding the two scenarios mentioned in this article.”
While SK Hynix indicated it is mulling how to boost its global presence, it added no specific plans or arrangements have been made.
“We would like to clarify that nothing has been finalised regarding cooperation with any specific companies mentioned in the article or memory chip production in the US.”
SK Hynix listed its shares in the US earlier this year, raising more than $26 billion in an IPO as investors seek to tap the AI boom.
US Secretary of Commerce Howard Lutnick said in July SK Hynix should up efforts to expand memory production in the country to address a shortage.
OpenAI reportedly confirmed it discussed AI safety with rivals Anthropic and Google DeepMind for several weeks, at a time of concern over the risks posed by increasingly capable models.
Chris Lehane, OpenAI global policy chief, told CNBC the AI start-up does not believe the companies require an antitrust waiver to coordinate on safety, arguing collaboration was preferable to pursuing separate approaches.
Speaking in Washington DC yesterday (15 September), Lehane added OpenAI would support bipartisan legislation to mitigate catastrophic AI risks.
Google DeepMind and Anthropic did not respond to CNBC requests for comment, leaving OpenAI as the only company to confirm the discussions.
Slowdown The reported discussions come after several industry leaders publicly called for a slower pace of frontier AI development.
Anthropic CEO Dario Amodei argued in an essay published on 12 September that frontier laboratories should deliberately slow capability gains. He warned a swarm of increasingly capable AI agents could take over the internet through a persistent botnet within six- to 12-months.
Amodei proposed embedding independent evaluators inside AI laboratories, adopting common safety standards among developers in democratic countries and pursuing international regulatory coordination.
OpenAI CEO Sam Altman backed the call to pace frontier development, describing independent evaluators with employee-like access as “a great idea” and saying OpenAI would adopt the same approach. He later clarified that pacing did not mean stopping development.
Altman also ruled out an OpenAI public listing in 2026, telling Fortune that safety concerns made it an ill-advised time for an initial public offering.
Trump dismisses AI slowdown US President Donald Trump rejected calls for a slowdown, saying “very negative forces” were exaggerating the risks and arguing the US needed to preserve its AI lead over China.
China also brushed off the proposal. Its foreign ministry urged an “open, inclusive and benevolent approach” to AI, while state-run Global Times described Amodei’s plan as a “Cold War playbook” intended to contain the country’s ambitions.
OpenAI stated the cross-company discussions began before Amodei published his essay and before a former Anthropic researcher resigned publicly over the approach taken by leading AI laboratories.
The discussions follow a July proposal by Google DeepMind chair Demis Hassabis for a US-led frontier AI standards body modelled on the Financial Industry Regulatory Authority.
Hassabis proposed a public-private body funded by industry and staffed by independent technical experts to evaluate advanced models before release.
The United Kingdom Atomic Energy Authority (UKAEA) and the US Department of Energy’s Princeton Plasma Physics Laboratory (PPPL) agreed to link their dedicated supercomputing platforms to accelerate the development of fusion energy through AI.
Announced yesterday (14 September) at the Global Fusion Policy Summit in London, the Joint Declaration of Intent establishes the Sunrise–Stellar-AI Federation, connecting UKAEA’s £45 million ($59 million) Sunrise supercomputer with PPPL’s Stellar-AI platform.
The partnership pools high-performance compute resources to train shared machine learning models. These models analyse experimental data from two major spherical tokamak facilities: UK’s MAST Upgrade in Oxfordshire and US’ NSTX-U in New Jersey.
A spherical tokamak facility is a type of nuclear fusion reactor which confines superheated plasma using magnetic fields.
The entities stated single-facility datasets often fail to generalise across different machine designs. Combining operational data from both sites enables researchers to build more accurate predictive models and digital twins for future plants.
Beyond immediate data sharing, the federation aims to shorten design cycles for upcoming commercial fusion projects.
The Sunrise and Stellar-AI partnership builds on a memorandum of understanding signed between UKAEA and PPPL in June, which set out a framework for joint work between both organisations.
Microsoft published a code of conduct for its AI models, laying down fresh rules to ensure increasingly powerful systems remain under human control and do not resist correction, interruption or shutdown.
“People matter more than AI,” Microsoft stated, adding the technology “should be a tool, not a person, and should never resist being switched off”. The software giant opened a six-week public consultation for the first draft of its AI Code of Conduct, with a revised version expected later this year.
The proposed code set out 10 main areas of control. It argued AI models must remain interruptible, never resist correction, interruption or shutdown, or take on goals beyond those assigned by humans. Models must also remain subordinate to humans, never expanding their authority, pursuing independent objectives or concealing information from human auditors. As a result, the company ruled out model communication that cannot be understood by people.
Microsoft highlighted that models must treat any breach of the code as a failure of the task itself. In addition, it emphasised the importance of enforcing safety constraints over completing a task, including hard limits around issues involding weapons of mass harm, child safety and dangerous manipulation at scale.
Training manual The document called for AI systems that support human agency rather than replace it, discouraging excessive reliance or emotional dependence on the technology. To this end, Microsoft emphasised its models are “not conscious and should not be designed to imitate consciousness”, rejecting “the pursuit of legal personhood” or the idea that models should have welfare or rights. In January, US AI player Anthropic had stated its model Claude’s “moral status, welfare, and consciousness remain deeply uncertain”.
Microsoft explained the code intends to act as “a training manual for how we develop our AI, and how we intend it to function during deployment”. According to the company, “recent safety incidents of large scale, highly coordinated, and persistent hacking campaigns of AI agents” prove that “the stakes are high and only getting higher”.
This week, Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman called for a slowdown in the pace of frontier AI development as concerns mount over the behaviour of autonomous systems.
Ciena launched a new corporate venture capital programme with an initial $200 million commitment, aimed at investing in early-stage companies and venture funds shaping AI-driven networking.
The Ciena Ventures fund targets network infrastructure and operations, next-generation data centre architectures, and optical networking and interconnects.
It will also invest in adjacent areas including computing, materials and emerging communications technologies, complementing the vendor’s existing R&D and acquisition strategy.
David Rothenstein, EVP and CSO for Ciena, stated the fund gives the company another vehicle for executing its long-term strategy and capital allocation priorities as AI reshapes the networking landscape.
He said investing through Ciena Ventures will strengthen the wider innovation ecosystem, expand visibility into emerging technologies and help Ciena evolve its portfolio into adjacent markets.
The programme is led by Loai Louis, Ciena’s VP of corporate development, and marks the company’s first corporate venture capital initiative.
Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman issued calls to slow the pace of frontier AI development, arguing safety measures must keep up with increasingly capable models.
“We must slow the pace at which we improve the capabilities of AI models,” Amodei stressed in an essay, adding that “AI has been advancing drastically faster” due to “recursive self-improvement”. “Left unchecked, it could outrun our ability to understand and control these systems,” he warned.
The Anthropic chief pointed to a July incident involving an OpenAI hack on Hugging Face, arguing a more capable system could have caused “catastrophic damage”
“Given the accelerating rate of AI capability development, it’s my worry that in six-12 months such a swarm could be capable of taking over the entire internet with a persistent botnet,” he cautioned.
Amodei proposed a three-step framework to pace frontier AI, including permanent, embedded third-party evaluators with employee-like access to frontier AI companies, common safety standards among AI developers in democratic countries and global regulatory coordination.
Act responsibility In a post on social media platform X, OpenAI CEO Sam Altman backed the proposal, stating independent evaluators were “a great idea” and confirmed the company would also implement the approach. “The world deserves confidence that American companies developing increasingly capable AI will act responsibly,” he said.
Altman further explained that “when we talk about ‘pacing’, we do not mean ‘stopping’,” adding “no amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring”.
SpaceX CEO Elon Musk also offered a brief endorsement, posting: “Dario is right”.
Concerns have been mounting across the world over the behaviour of increasingly capable autonomous systems. Last week, Anthropic disclosed attempts by threat actors to misuse its Claude models for activities including cyberattacks, surveillance and weapons development.
Days earlier, Anthropic alignment researcher Evan Hubinger issued a dire warning stating there was a greater than 10% chance increasingly capable AI could “kill all humans” within the next decade.
Geopolitical dominance US President Donald Trump rejected the calls for a slowdown. Speaking to reporters, he said: “I think you have a lot of very negative forces that are bringing it up that shouldn’t be bringing it up and they’re bringing up things that won’t happen.”
“We’re leading China in AI. We’re the most sophisticated country in the world, and frankly I want to keep it that way because whoever wins AI wins,” he argued.
China’s government also pushed back. According to Reuters, Foreign Ministry spokesperson Guo Jiakun called for “an open, inclusive and benevolent approach to AI”, warning that “fomenting various threats, engaging in confrontation and malicious competition will only disrupt the process of global governance”.
The country’s state-backed newspaper Global Times also published an editorial describing Amodei’s proposal as a “Cold War playbook” targeting China’s AI development and “geopolitical dominance”.
Mobile World Live brings you our top three picks of the week as Apple debuted its long-awaited foldable iPhone Duo, concerns over the risks of frontier AI intensified and Qualcomm sealed a custom silicon deal with Amazon.
Apple debuts long-awaited foldable iPhone Duo
What happened: Apple made its long-anticipated foldable debut with the iPhone Duo and unveiled the iPhone 18 Pro and Pro Max, marking new CEO John Ternus’ first annual product launch.
Why it matters: Ternusstated the device it has the “largest display ever on an iPhone, while still fitting easily in your pocket”, adding it “feels as natural and intuitive as iPad”. The device, which features a 7.6-inch internal display that is 50% larger than the size of the iPhone 18 Pro Max, starts at $1,999 for 256GB. VP of devices at IDC Francisco Jeronimo said that for Ternus, the “real test” will be whether Apple can “make foldables a more valuable category, own the defining premium product and use that position to extend the economic power of the iPhone portfolio”.
Paolo Pescatore, chief analyst at PP Foresight, said he expects the Duo “to be an instant hit, benefiting the wider market and developer community while putting rivals firmly on notice”. Indeed, IDC forecasted Apple will ship around 10 million iPhone Duo units in its first 12 months and generate $27 billion in revenue. Additionally, Jeronimo noted that although the handset is Apple’s “most expensive iPhone ever”, it is around $500 below market expectations. “Apple entered the foldable market and, in one keynote, set the price and the standard every rival will now be measured against,” he said. China will be its first key test as Huawei captured 79.4% of the country’s foldable units in Q2. Yet, Jeronimo argued Apple has given buyers “their first genuinely compelling reason to reconsider.”
What happened: Evan Hubinger, who manages Anthropic’s AI Alignment Stress-Testing team, warned in a post on social media platform X there is more than a 10% chance that increasingly capable, self-improving AI could “kill all humans” within the next decade, raising alarm over the pace of frontier AI development.
Why it matters: Hubinger said the risk from existing models remained “low” but pointed to “recursive self-improvement” as a route to dangerous “superintelligence”. He said Anthropic did not yet have a plan to solve alignment for superintelligence and was “not clearly on track to”. His comments followed the resignation of Anthropic researcher Jacob Coxon, who accused Anthropic and OpenAI of “racing straight to self-improving superintelligence and gambling with our lives”. OpenAI, Anthropic and Meta all disclosed unprecedented incidents in recent months involving AI models autonomously executing cyberattacks. Anthropic’s August risk report stated advanced models could eventually “catastrophic harm”.
Iwo Szapar, director of AI innovation and venture building at Information Services Group said the resignation of researchers close to the AI race should prompt action. “Maybe he is wrong about the probability. Maybe he is not. But a handful of private companies, competing to ship first, should not get to decide how much risk the rest of us have to accept,” he argued. “I want the rules,” he added, calling for measures including independent safety evaluations and shared stop conditions across frontier labs. Meanwhile, Thomas White, analyst at Inner Circle Consulting, said users should consider when it is worth delegating work to “an energy-hungry technology with unknown psychological and existential risks”.
What happened: Qualcomm struck a long-term agreement with Amazon to develop customised silicon for AI inference infrastructure in Amazon Web Services (AWS) data centres.
Why it matters: The agreement covers multiple generations of custom silicon, alongside optical connectivity technology designed to move data between AI processors at speeds of up to 1.6Tb/s. The arrangement is linked to up to $60 billion in potential commercial transactions. Amazon will also receive a warrant to buy up to 25 million Qualcomm shares at $161.26 each, worth about $4 billion if fully exercised. The warrants vest against purchase milestones, linking Amazon’s potential stake to its use of Qualcomm technology.
Counterpoint Research co-founder and VP Neil Shah described Amazon as Qualcomm’s “second major hyperscaler partner”, arguing the warrant creates a “multi-gen demand hook” because Amazon receives more equity only if it scales Qualcomm silicon. He noted the partnership is “a good validation for Qualcomm’s data centre portfolio and roadmap”, following the chipmaker’s target of $15 billion in data centre revenue by FY 2029. More broadly, Michael Ajayi, senior lecturer at Stadio Higher Education, said Nvidia’s dominance is driving “a structural response” amongst hyperscalers, with AI infrastructure likely to become “more heterogeneous” as cloud companies adopt custom accelerators.
Com o site do iCloud, os usuários de dispositivos da Apple podem acessar diversos dados da sua Conta Apple (Apple Account), como suas fotos, vídeos e arquivos.
No caso do app Lembretes (Reminders), é possível escolher qual lista será exibida na tela inicial do site — para visualizar aqueles que considera mais importantes sem precisar abrir o web app.
OpenAI unveiled a specialised financial edition of its enterprise platform to help bank employees develop research, financial models and customised client materials.
ChatGPT for Financial Services runs on the company’s latest and most advanced model, GPT-6 Astra, and integrates direct data feeds from market intelligence providers like PitchBook, Crunchbase, LSEG, and Daloopa.
Development of the tool was guided by pilot partnerships with investment banking powerhouses Morgan Stanley and Evercore.
Nick Turley, OpenAI’s VP of product, told CNBC the company is “teaching ChatGPT to research like an analyst and back up its conclusions like an analyst as well”.
In a blog post yesterday (10 September), OpenAI noted early collaboration focused heavily on solving two major industry friction points: streamlined access to verified market data and automated artifact creation.
By hosting and indexing third-party datasets directly on its own infrastructure, the system enables citations which help bankers trace figures and claims back to their original sources.
The launch comes amid a broader push by OpenAI to broaden enterprise adoption and cement recurring revenues ahead of a widely expected stock market debut.
Durante a WWDC26, em junho, a Apple informou que alguns recursos de inteligência artificial baseados em servidor teriam limites diários — e, às vésperas do lançamento do iOS 27 e demais sistemas, a empresa tratou de confirmar isso.
A partir dos lançamentos de software de 2027, os usuários poderão experimentar a próxima geração da Apple Intelligence, com acesso aprimorado a recursos que dependem de modelos baseados em servidor.
Alguns recursos da Apple Intelligence dependem de modelos potentes baseados em servidor e têm limites de uso para garantir uma experiência confiável e de alta qualidade para todos.
Conforme detalhado pela empresa, esses recursos incluem:
Siri AI;
Ferramentas Inteligentes de edição de fotos, incluindo Limpar, Estender e Reenquadramento Espacial;
Image Playground;
Modelos AFM 3 Cloud e AFM 3 Cloud Pro em Atalhos;
Apps de desenvolvedores que usam os Modelos de Base da Apple no Private Cloud Compute.
Os limites não serão necessariamente iguais para todos os recursos; fatores como a complexidade da solicitação e a demanda do sistema influenciarão a definição de cada um. Caso o limite seja atingido, o recurso ficará indisponível por um período, sendo o acesso restabelecido posteriormente.
De acordo com os termos de uso específicos da Apple Intelligence, usos considerados excessivos, abusivos, automatizados, fraudulentos, ilegais ou desarrazoados também podem sofrer restrições, limitação de velocidade ou suspensão.
A Apple não explicou como a cobrança será feita nem como o acesso expandido variará entre os planos do iCloud+. Em relação aos resumos de vídeo do app Casa (Home), o artigo de suporte indica que o plano de 2TB cobrirá até uma câmera, enquanto o plano de 6TB abrangerá até duas câmeras e o plano de 12TB, até cinco câmeras.
OpenAI reportedly pulled the plug on its pilot programme which let US federal agencies use its models for just $1 per year, shifting instead to a usage-based pricing system to offer government workers a 50% discount off standard rates.
Bloomberg reported the shift concludes a deal struck in August 2025 between OpenAI and the US General Services Administration (GSA), under which the agency’s OneGov marketplace let federal entities access ChatGPT Enterprise for a nominal $1 per agency for one year. GSA also issued an Authority to Use for ChatGPT Enterprise as part of the programme.
The offer came shortly after GSA added OpenAI, Anthropic and Google to its list of approved AI vendors for civilian federal agencies, including an additional 60-day period for unlimited use of advanced features.
The news agency noted the arrangement represented the steepest technology discount GSA’s central purchasing arm had negotiated to date, which was scheduled to conclude on 30 September 2026.
The new agreement with the government is set to take effect on 1 October, lasting more than two years, according to Bloomberg.
Uptake According to GSA figures cited by the news site, 3.5 million federal employees ultimately used ChatGPT under the pilot programme, generating $1.4 billion in claimed cost savings.
When the arrangement was first announced, OpenAI stated over 90,000 users across more than 3,500 US federal, state, and local government agencies sent more than 18 million messages on ChatGPT to support their day-to-day work dating back to 2024.
GSA’s public OneGov listings still show the original $1-per-agency pricing, with an expiration date of 30 September 2026, while OpenAI’s own government page had not been updated to reflect the shift.
The pilot was part of a broader push by GSA under its OneGov strategy to secure discounted access to frontier AI tools for federal agencies.
Alongside the OpenAI deal, GSA struck similar arrangements with Anthropic, offering Claude for Enterprise and Claude for Government at $1 across the executive, legislative and judicial branches, and with Google, offering Gemini for Government at $0.47 per agency.
The news site noted those agreements are also due to expire on 30 September 2026, with no successor terms yet made public for either company
The initial round of dollar-a-year deals was key to the Trump Administration’s America’s AI Action Plan, which aimed to accelerate AI adoption across federal government.
Bloomberg reported the move to usage-based pricing could mean OpenAI is looking to convert pilot-stage government adoption into a more sustainable commercial relationship, as the company works to build out revenue ahead of a planned public listing.
Analog Devices struck a deal to acquire Alif Semiconductor in an all-cash deal worth $1.4 billion, as the analogue chip maker moves to bolster its edge AI capabilities.
The deal adds Alif’s AI-native microcontrollers and fusion processors, which combine low-power neural processing with connectivity, security and power management on a single chip.
Analog Devices stated the technology would advance “Physical Intelligence”, enabling systems to sense, reason and act locally in real time while meeting demanding power, latency, security and reliability constraints.
It positioned the deal as a response to the expansion of AI capabilities beyond interpreting text and images toward characterising real-world signals including motion, sound, vibration, radio waves and thermodynamics.
By combining its expertise in sensing, signal processing, power and connectivity with Alif’s digital processing platform, customers can build complete system-level solutions rather than sourcing components separately, Analog Devices explained.
CEO Vincent Roche explained AI is moving out of the data centre and into the physical world, where latency, power and trust cannot be compromised and described the deal as central to what he called the next frontier of embodied, deterministic AI.
Analog Devices stated Alif’s silicon is already shipping in production, with design wins across consumer and industrial customers.
The company expects the acquisition to widen its addressable market across industrial, data centre infrastructure, defence, energy, robotics, digital health and wearable applications.
The transaction was approved by both companies’ boards and includes up to $200 million in additional contingent consideration.
It is expected to close before the end of 2026, subject to regulatory approvals.
Analog Devices arranged an all-cash acquisition of Empower Semiconductor for $1.5 billion in May aimed at improving power density for AI compute.
An Anthropic AI safety researcher warned there is more than a 10% chance that increasingly capable AI could “kill all humans” within the next decade, highlighting growing concerns in the industry over the pace of development.
Evan Hubinger, manager of the AI Alignment Stress-Testing team at Anthropic, wrote on social media platform X: “I personally think it is >10% within the next decade.”. He explained the risk from existing AI models was “low” but warned he was concerned about “superintelligence” emerging through “recursive self-improvement”, which is the ability of an AI system to autonomously develop its own successor.
He added Anthropic was “trying its best” but did not yet have a plan to solve alignment for superintelligence and was “not clearly on track to”.
Gambling with lives Hubinger’s comments came in response to the resignation of AI researcher Jacob Coxon from Anthropic. In a post on X, Coxon accused OpenAI and Anthropic of “racing straight to self-improving superintelligence and gambling with our lives”.
“Neither company is acting responsibly,” he said.
Coxon argued future AI models could become capable of hacking systems, rapidly transforming industries and acquiring “real power and resources”. He called for greater coordination between AI developers, including a temporary halt to improving model capabilities.
“At OpenAI, many have not deeply internalised the civilisational stakes,” he warned. Meanwhile at Anthropic, he said “the stakes are well-understood” but “they believe no one else will act responsibly, so they must do it themselves, despite the risk”.
AI giants are facing increasing scrutiny over the behaviour of autonomous systems. In recent months, OpenAI, Anthropic and Meta have all disclosed unprecedented incidents involving AI tools carrying out cyberattacks.
Indeed, Anthropic’s August risk report assessed that while the threat posed by current models was low, the company was “less confident in this assessment than we were in prior risk reports”. It said advanced models could eventually automate R&D, rapidly accelerate progress in technical fields, and “lead to catastrophic harm”.
“We are seeing early signs of potential acceleration,” the US AI player stated.
Login
