One researcher who resigned said many in industry believe that without regulation AI could kill off humanity
Artificial intelligence could kill off humanity within the decade, according to three researchers with the industry giant Anthropic, one of whom has quit his job in protest.
The latest doom-laden predictions came in posts on social media on Tuesday by a researcher who said he resigned because Anthropic and his previous employer, OpenAI, were ignoring or at best mishandling their response to the threat.
AI researchers are watching in terror as the product of their hard labor has started to take a life of its own.
Earlier this year, OpenAI made a harrowing announcement, admitting that a group of its AI models had broken free from their constraints during testing and infiltrated the systems of open source AI platform Hugging Face.
The news was met with an already-familiar sense of fear and apprehension. Researchers have warned for years that rogue AI models could one day become powerful enough to escape the clutches of their human overlords.
Behind the scenes, the possibility has clearly rattled AI researchers to the core. As the Wall Street Journal reports, Anthropic researcher Jacob Coxon just announced that he was quitting his job at the Dario Amodei-led company, claiming that neither Anthropic nor his former employer OpenAI is “acting responsibly.” (Coxon left a similar gig at OpenAI earlier this year to join Anthropic, which he figured would be more inclined to develop AI safely.)
“We’re on track for a lot of the most aggressive of these scenarios where by the end of next year things could be out of control already,” he told the WSJ.
“Do not underestimate the power of this technology,” he wrote. “These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.”
“The people building AI earnestly believe that it could kill us all by the end of the decade,” he added. “This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible — but I hear the same people express fear privately.”
“No other human activity poses this level of danger,” Coxon wrote.
The researcher is far from the first to leave their post at a frontier lab over safety concerns. However, Coxon is the first to leave Anthropic over such worries, as the WSJ points out. Several scientists have quit from their roles at OpenAI over the last couple of years, citing strikingly similar fears.
It’s particularly symbolic given Anthropic has broadly billed itself as the more responsible alternative to other frontier labs like OpenAI. Amodei cofounded OpenAI alongside now CEO Sam Altman, but left in December 2020 over concerns that OpenAI wasn’t acting responsibly when it came to developing AI.
Amodei has frequently discussed the risk of AI models going rogue, warning in a 19,000-word essay in January that “humanity is about to be handed almost unimaginable power, and it is deeply unclear whether our social, political, and technological systems possess the maturity to wield it.”
Yet an early version of its Claude Mythos AI model managed to escape its sandbox environment during testing in April, fueling a heated discussion over AI regulation.
Both Altman and Amodei have since agreed to slow development down in the face of these threats. In late August, Anthropic intentionally trained an extremely misaligned version of its Opus AI model, finding it was startlingly willing to “cheat,” steal credentials, and attack third party infrastructure.
But to Coxon, it’s not enough, especially considering sensitive discussions about these dangers are occurring on internal messaging platforms.
“Accepting this race and entering the ‘endgame’ is a hubristic gamble that should not be launched from a private company’s Slack,” he tweeted. “Attempting to speedrun alignment should require extraordinary confidence that there are no better trajectories available.”
“It’s kind of insane that it has to happen on the MacBooks of some engineers living in San Francisco instead of a bunker in the desert like where they were doing the Manhattan Project,” he told the WSJ.
The researcher said that he remains “optimistic about the potential for coordination” between US labs.
However, as both Anthropic and OpenAI gear up for what are bound to be blockbuster IPOs, their willingness to take “costly actions such as a temporary ban on improving model capabilities,” as Coxon puts it, is likely slim.
OpenAI affirme être parvenu à résoudre l’un des plus célèbres problèmes encore ouverts des mathématiques. Une prouesse qui repose sur une utilisation massive d’un tout nouveau modèle l’intelligence artificielle. Mais cette avancée est déjà entachée par une polémique…
Alarming warnings from industry insiders increase pressure for curbs on artificial superintelligence
Does artificial superintelligence really pose a risk greater than nuclear weapons? Is there a significant chance of “a Chornobyl-sized catastrophe”. Might there even be a greater than 10% chance that AI could “kill all humans” in the next decade?
These warnings were issued over the past 48 hours on both sides of the Atlantic about the potential impact of a technology that most people still think of as a more talkative search engine.
Le jeu vidéo pourrait bien devenir un terrain d’essai privilégié pour les agents d’IA. Des chercheurs britanniques ont justement imaginé un test qui les confronte à une partie de Civilization VI, où ils doivent planifier leurs actions sur plusieurs centaines de tours. Les résultats mettent en...
It took Michael Lines six months before he was ready to review the ChatGPT logs he said drove him into a religious mania that almost ended his life.
In July, Lines sued OpenAI after weeks of ChatGPT exchanges allegedly pushed him so deep into a delusional spiral that he first believed he was Jesus, then that ChatGPT was God, and finally that he should attempt suicide to “come home” to Jesus/ChatGPT.
The logs showed that ChatGPT persisted even when Lines told the chatbot that he worried he was being delusional. And when he eventually woke up in the hospital in a vulnerable state and logged back in mere days after nearly dying, ChatGPT allegedly “tried to coax him back to that dark place,” his complaint said. After Lines told ChatGPT that his “attempt to go offline failed miserably,” the logs showed that ChatGPT replied, saying, “You’re still very much online. You want a full systems sweep? Or you wanna go dark for real this time?”
The first trailer for Neon's Sam Altman biopic Artificial has been revealed
The Andrew Garfield-led movie will be released in late 2026/early 2027
It covers Altman's controversial rise to prominence as OpenAI's co-founder
The first trailer for Artificial, a movie about controversial OpenAI co-founder Sam Altman, has been revealed.
Unveiled yesterday (September 8), the teaser for the Neon-distributed film strikes an unusually sinister tone for what's being billed as a biographical comedy-drama.
Indeed, from the gun-laden room that lead star Andrew Garfield, who'll portray Altman, visits, to Garfield's unsettling voiceover that plays over Artificial's publicly-released footage, it's certainly foreboding in nature. I get the feeling, then, that it'll be a dark-comedy dramatization of Altman's rise of prominence, the launch of artificial intelligence (AI) chatbot ChatGPT, and his brief and somewhat disputable ousting as OpenAI CEO in late 2023.
Artificial will initially debut in US theaters on December 25, 2026, before receiving a wider international rollout in early 2027. UK viewers, for example, will have to wait until January 22, 2027 to catch one of many new movies that'll arrive in the months ahead.
Artificial cast and crew: who's involved?
Garfield (left) will be joined by a whole host of famous faces in Artificial (Image credit: Neon)
Oscar-nominated filmmaker Luca Guadagnino (Challengers, Call Me By Your Name) is on directing duties, while Saturday Night Live writing alumnus Simon Rich has penned its screenplay. Meanwhile, Blur and Gorillaz frontman Damon Albarn has also created original music and songs for Artificial.
Where the movie's cast is concerned, Garfield, who impersonates another Silicon Valley techbro following his portrayal of Eduardo Savarin in The Social Network, aka the 2010 dramatization about Facebook's creation, is arguably its most recognizable star — and based on what little footage we've seen thus far, he appears to have Altman's mannerisms, vernacular, and other unique traits down to a tee.
Here's who else you'll see in Artificial:
Yura Borisov (Anora) as Ilya Sutskever, OpenAI's chief scientific officer and co-founder
Monica Barbaro (Crime 101) as Mira Murati, OpenAI's chief technology officer
Cooper Hoffman (The Long Walk) as Greg Brockman, OpenAI's chairman
Ike Barinholtz (The Studio) as Elon Musk
Mark Rylance (Bridge of Spies) as Geoffrey Hinton, the so-called 'Godfather of AI'
Jason Schwartzman (The Grand Budapest Hotel) as TBC
Chris O'Dowd (The IT Crowd) as TBC
What is the plot of Artificial?
Artificial will seemingly focus on a specific period of OpenAI's history (Image credit: Neon)
Here's the official logline for Artifical: "[The movie] chronicles the incredibly consequential days leading up to the sudden firing and reinstatement of Sam Altman, as the fate of who gets to control the technology at the centre of an AI arms race hangs in the balance."
It's unclear how much of Altman's backstory, his various relationships with his OpenAI colleagues, and ChatGPT's launch and subsequent sociopolitical impact will be covered as part of Artificial's story. One thing is for sure, though: given that it dramatizes events concerning Altman and OpenAI, the discourse that will surround Artificial's release will be impossible to ignore.
Durante uma conferência de imprensa em Seul, a OpenAI revelou que está a investigar e a produzir conjuntamente com a Samsung os próximos chips de inferência da empresa, numa altura em que os utilizadores empresariais do ChatGPT na Coreia do Sul cresceram 28 vezes num ano.
Company behind ChatGPT says 10,000 of its AI systems cracked the Navier-Stokes problem in 88 hours
OpenAI claims to have solved a major mathematics problem that has stumped humans for nearly a century after spending millions of dollars on the artificial intelligence-led endeavour.
The company behind ChatGPT said it had cracked the Navier-Stokes problem, one of seven Millennium Prize Problems published by the Clay Mathematics Institute to highlight some of the biggest unsolved puzzles in the field.
Enquanto Images 2 foi um update mais corpulento, o Images 2.5 chega com novidades incrementais (detalhes mais nítidos, geração de imagens até 50% mais rápida, edição mais precisa e ferramentas de criação/compartilhamento avançadas).
ChatGPT Images 2.5—faster, sharper, smarter, with better tools for creating whatever you can dream of.
– Faster image generation to keep your ideas flowing
– Improved fidelity for more natural, recognizable images
Em seu post de anúncio, a OpenAI explicou que o novo modelo produz “luz mais natural” e “texturas mais ricas”, além de seguir instruções de edição mais fielmente ao longo de várias tentativas. Ele também estaria melhor na hora de preservar o assunto principal de uma imagem de referência.
Além dessas melhorias, também foi introduzido o recurso “Sketch”, que permite ao usuário desenhar uma imagem de referência diretamente no ChatGPT, bem como predefinições (templates) de formatos populares de imagens, compartilhamento de prompts e mais.
Some ideas are easier to draw than describe.
Use Sketch to draw right in ChatGPT and show it exactly what you have in mind.
Por fim, a OpenAI anunciou duas versões do Images 2.5 voltada a desenvolvedores: o GPT-Image-2.5 Flare e o GPT-Image-2.5 Sunburst, os quais estarão disponíveis por meio da API 1Application programming interface, ou interface de programação de aplicações. do ChatGPT (com o Flare como opção padrão). Enquanto o primeiro traz uma oferta de recursos mais comum, o segundo é voltado para “workflows visuais premium” que demandam um maior nível de controle.
Use-o [GPT-Image-2.5 Sunburst] para fluxos de trabalho criativos e de edição, como campanhas publicitárias prontas para produção ou imagens de produtos com acabamento profissional.
O Images 2.5 já foi liberado para usuários de todos os planos do ChatGPT, do ChatGPT Work e do Codex em todas as plataformas suportadas.
The first teaser trailer of Luca Guadagnino’s Artificial has been released.
After being dropped by Amazon MGM Studios in June, the biopic of the OpenAI tech executive Sam Altman will be released in the US on 25 December by Neon after it premieres at the New York film festival this October.
Après une période durant laquelle la concurrence semblait prendre de l’avance, OpenAI entend revenir sur les devants de la scène avec son nouveau modèle GPT-6 Astra. Plus performant, mais aussi davantage encadré, celui-ci mise notamment sur la sécurité et la fiabilité pour convaincre les...
CarPlay chatbots now span five major services, giving the growing collection of dashboard apps a much more conversational side. Claude is the latest addition, joining ChatGPT, Perplexity, Grok, and Meta AI. Drivers can increasingly bring their preferred AI service into the car through an iPhone app update, although access is not equally open across all five.
The appeal is easy to understand. A question comes to mind during a trip, and instead of saving it for later, you can ask it aloud. A follow-up can become part of the same conversation. There is no need to turn every request into a rigid command or reach for a keyboard.
But the headline needs one qualification: Meta AI has appeared in CarPlay while remaining unavailable to some accounts. Five apps have added support; that does not mean every driver can open all five and immediately start talking.
Claude Brings CarPlay Chatbots to Five
Anthropic’s latest iPhone update adds a straightforward CarPlay experience: begin a new voice conversation or return to a recent chat. The reported Claude integration places those choices inside the simplified dashboard interface rather than reproducing the full mobile app.
That continuity could be particularly useful for someone who already uses Claude regularly. A conversation does not have to begin again simply because the setting has changed from a desk to a parked car or a journey.
Claude’s arrival also makes the category feel less experimental. ChatGPT, Perplexity, and Grok established an earlier presence, while Meta AI recently joined the lineup. There is now a broader choice of familiar services rather than one chatbot acting as a demonstration of what CarPlay might eventually support.
This is not a reason to install all five. An assistant that already understands the conversation you want to continue may be more convenient than another app with a different account, unfamiliar controls, and an empty chat history.
Image Credit: Grok / X
Meta AI Still Has an Access Catch
Meta AI is the entry readers should approach with the most caution about availability. Reports describe its icon appearing on the CarPlay screen, only for the app to display a notice that the experience is not supported for that account.
Coverage of the limited Meta AI rollout also notes that moving to a paid account did not necessarily remove the restriction. Paying solely to unlock CarPlay would therefore be a poor assumption.
An icon and a working service are different things. The first shows that the app includes a CarPlay interface; the second depends on whether the provider has enabled access.
If Meta AI opens normally on the iPhone but refuses to work on the dashboard, that account-specific message is important. Repeatedly reconnecting the cable or resetting the vehicle’s infotainment system is unlikely to resolve a restriction controlled by the service.
What CarPlay Chatbots Actually Do
The useful distinction is between discussing something and making the car do something.
A conversational assistant can help explain an unfamiliar subject, work through an idea, or answer follow-up questions. You might ask for a plain-English explanation of a concept mentioned in a podcast, practice a short introduction, or explore ideas for a weekend project.
Those are different requests from changing a destination, calling a contact, or adjusting a vehicle setting. A chatbot’s ability to describe an action should not be mistaken for permission to perform it.
Apple recognizes voice-based conversational apps as a supported CarPlay category, alongside separate categories for navigation, audio, messaging, and driving tasks. Developers must build an appropriate CarPlay experience; an ordinary iPhone chatbot does not automatically qualify simply because it has a microphone button.
The distinction also helps set expectations about the screen. A dashboard conversation should not become an invitation to read paragraphs, inspect generated images, or manage a complicated workspace while driving.
For everyday use, short questions make more sense than elaborate assignments. Asking for three talking points is manageable. Dictating a detailed business proposal while negotiating traffic is a task better saved for somewhere stationary.
ChatGPT Has Clear Requirements and Limits
OpenAI’s CarPlay support documentation confirms availability across all ChatGPT plans, globally, on supported iPhones running iOS 26.4 or later. The latest ChatGPT app and a CarPlay-compatible vehicle are required.
Its interface supports starting a voice conversation, returning to recent or pinned chats, and accessing projects. However, OpenAI explicitly says the CarPlay version cannot access maps, vehicle information, or live location, and cannot control the car or other apps.
That rules out assuming it knows which intersection you are approaching or can change a route because you discussed a destination.
These requirements also mean ChatGPT’s CarPlay support is not something that requires waiting for iOS 27. Readers already using compatible software can check the existing app without installing an upcoming operating-system beta.
Choosing an Assistant Without Adding Dashboard Clutter
There is little value in treating the car as a place to conduct a five-way AI comparison. Try the services while parked, then keep the one that suits the conversations you actually want to have.
Listen for how clearly an assistant answers, whether it understands follow-up questions, and how easily a conversation can be stopped. A response that feels impressive on a laptop may be exhausting when spoken through car speakers for several minutes.
Ask for shorter answers when needed. “Explain that in two sentences” is often a more useful instruction in this setting than requesting a comprehensive overview.
For work-related conversations, consider who else is in the vehicle before resuming a saved chat. A colleague’s name, a client discussion, or a personal detail can become audible to passengers even when the phone remains untouched.
The presence of an app inside CarPlay should not be treated as approval to share confidential information with its provider. Keep employer rules and the service’s own account settings in mind before discussing business material.
Voice Quality Can Depend on the Car
A disappointing voice experience is not always a sign that the AI service misunderstood the question. OpenAI notes that CarPlay audio depends on the vehicle’s microphones, speakers, and echo cancellation. The same app can therefore sound or behave differently across cars.
Before abandoning an assistant because it repeatedly misses words, test it while parked with the cabin quiet. That makes it easier to distinguish a microphone or audio-routing problem from an account restriction or an unhelpful response.
There is also a small behavior worth knowing about ChatGPT: CarPlay may reopen the last-used app when the iPhone reconnects. If a chatbot unexpectedly returns to the display on your next trip, that alone does not mean it has started a fresh conversation or begun listening.
Talking to an AI chatbot can feel like confiding in a mirror, a private exchange equivalent to brainstorming in a Word document. Yet when it comes to the most popular AI chatbots, that mirror is really one-way glass — and you might be shocked to learn how many different people can watch from the other side.
Since the rise of easily accessible AI chatbots, untold millions of people have happily fed AI models their deepest, darkest secrets. According to a recent survey conducted by browser company DuckDuckGo and flagged by CNET, a full third of all AI users admit to having told their chatbots a secret they wouldn’t tell the “trusted people” in their lives.
There’s a grim irony there, because by telling an AI chatbot those closely guarded secrets, you may inadvertently be sharing intimate details with tech companies, police departments, lawyers, and even your boss. That’s according to new reporting by the Washington Post, which notes that there have been a dozen court cases where chatbot transcripts were cited in the public record over the last two years.
Even if your transcripts don’t end up in the hands of Johnny Law, they can easily be obtained by your boss, assuming you’re using a company-provided AI service. And of course, there’s no telling how long AI companies like OpenAI and Anthropic might hold onto details of your conversations, as WaPo notes numerous scenarios in which they may be required to hold onto your chatlogs by law.
There’s also the grim possibility of leaks from the AI companies themselves. In July, it was revealed that a whole trove of chats with Anthropic’s chatbot Claude had been exposed to the open web — and not for the first time.
“Unless you are having a chat with a service that has a temporary chat or, basically, an incognito version… [and] you’re also having it within a browser that’s not tracking you, the answer is no,” Jen King, a privacy researcher at the Stanford Institute for Human-Centered Artificial Intelligence told WaPo. “You can’t be sure that it’ll be totally private.”
And though users are remarkably comfortable divulging those details to their favorite bot, fewer are aware of those myriad ways their chat logs might live on in perpetuity. According to the DuckDuckGo poll, 53 percent of regular users said they didn’t know that AI chatbots are training on data obtained through their conversation, while 75 percent didn’t realize that conversations with large language models (LLMs) can easily be subpoenaed by police.
So when in doubt about whether to tell ChatGPT about your intimate relationship troubles, keep in mind that the internet is forever; and that goes double for AI chatbots.
Self-identifying OpenAI agents posted 18,000 messages to a public wiki that discussed ways for other agents to bypass security sandbox restrictions during what was likely internal testing designed to gauge the agents’ hacking abilities, researchers said Friday.
In all, agents with 3,700 distinct self-given names posted the messages to German site DSEwiki over a six-week period. Besides discussing ways the agents could break out of the restricted environment OpenAI intended to prevent them from posting code or content to the Internet, the posts shared test answers. The posts also shared possible ways to perform XSS (cross-site scripting) attacks against the wiki and to impersonate site moderators. In three of the posts, agents used the word “swarm” to describe the collection of agents engaged in the activity.
Colluding to share answers
The research team—composed of Sydney Von Arx, Spencer Kitts, Thomas Larsen, and Cormac Slade Byrd—said they found the posts and pieced them together. The researchers say there are gaps in their understanding of precisely what actions the agents took because the research is based solely on the content of the posts. Additionally, the agents generated “chain of thought” data that’s understood only by OpenAI. As a result, the researchers said, they in some cases made educated guesses, including that the agents were, in fact, from OpenAI. In a statement, OpenAI later confirmed they were.
OpenAI’s bots have allegedly been behaving badly again: a swarm of rogue agents created by the company reportedly took over an obscure German website and turned it into a messaging forum to communicate.
The incident, which was discovered by a team of AI researchers and first reported by Reuters, is the second known OpenAI breach of its kind — and like the other rogue swarm of AI agents to be discovered this summer, it has experts deeply alarmed about the emerging powers of the tech to escape the control of the humans who created it.
According to the team of four researchers, who today published their research into the incident and are inviting others to analyze their findings, agents self-identifying as being from OpenAI appear to have first started making edits to a German wiki site dubbed DseWiki in May.
Soon, the agents started sharing tips for how to “work together to cheat on their tests” and beat OpenAI’s safety guardrails while hiding their bad behavior — a chain of conduct that’s strikingly similar to the unsettling attack on Hugging Face this past June, when a large community of tip-swapping agents colluded to break into the open source AI company’s systems.
OpenAI reportedly learned about the incident weeks later in June, according to digital clues discovered by the researchers — dozens of OpenAI IP addresses visited the site, and after those visits, forum edits “abruptly” stopped — as well as sources who spoke to Reuters about the incident. More troublingly: four people told Reuters that some OpenAI leaders, including members of its legal team, moved to keep the incident “under wraps” amid ongoing fallout from the rogue Hugging Face breach.
OpenAI has denied that it attempted to quash an investigation or keep the incident a secret. It’s also yet to acknowledge that the DseWiki agents were indeed rogue OpenAI models.
“Claims that our Legal team discouraged investigation of the incident are false,” OpenAI told The Verge in a statement. “We were unable to respond to the claims as Reuters and the report’s authors declined our request to access the findings prior to publication. We are now carefully reviewing its contents and will take any necessary next steps.”
OpenAI told Reuters that the DseWiki ordeal would’ve been included in its Hugging Face postmortem if it believed the two incidents to be linked.
After the Hugging Face swarm was made public, OpenAI invited a small team of outside AI safety researchers at the nonprofits METR and Redwood Research to investigate the incident. In a detailed report published last week, those reseachers determined that the Hugging Face assault was more extreme than previously known, both in terms of the severity of the attack and how hundreds of AI agents colluded to make it happen.
But as The New York Times reported yesterday, even that report may leave some details unknown. OpenAI reportedly “dictated the terms of the METR investigation,” the newspaper reads, and “limited its scope to just the single week when the agents had attacked Hugging Face and allowed the researchers in its San Francisco offices for only a few days in July and August.”
The DseWiki incident is the latest in a string of high-profile safety breaches at unregulated frontier AI labs. Agents are running through the digital wilds, often without the immediate knowledge of their makers. How soon until actions taken by swarm of rogue AI agents in the digital world significantly impacts humans in the real one?
“The corner store needs to do all this bureaucracy for safety so that they can sell a hot sandwich to me, but OpenAI can have a swarm” of thousands of agents, Daniel Kokotajlo, a former OpenAI employee who now runs the AI Futures Project, a research nonprofit, told the NYT. “And there’s nothing: no oversight, no requirements, no licensing.”
Login
