Yesterday, Amir and I examined the shock waves that have rippled through the AI industry and far beyond since former Anthropic researcher Jacob Coxon very publicly quit his job over concerns that AI “could kill us all”—and another Anthropic AI safety specialist put the probability of such a catastrophe at greater than 10%.

Coxon told me that is hardly the most pessimistic view inside Anthropic. “People have varying probabilities that—barring some substantial coordinated slowdown—the whole thing ends in doom,” he said in an interview later Wednesday. Those estimates range higher than 50% among some Anthropic employees, he said.

Coxon was able to gauge the fears of his fellow employees because “Anthropic has a substantially more transparent internal culture than OpenAI,” where Coxon worked for years before joining Anthropic in May.

Coxon, who focused on the earliest stages of training AI models at Anthropic, said he decided to leave the company because he was gradually “starting to viscerally feel the fear of where the tech's going to be like the next two years or even the next one year.”

An Anthropic safety researcher’s warning Tuesday that his company believes AI “could kill all humans” within a decade has reverberated throughout Silicon Valley, Wall Street and Washington, potentially complicating the Claude maker’s upcoming initial public offering and its efforts to repair ties with U.S. political leaders.

As the comment circulated around the internet and on TV broadcasts, some Anthropic employees and other AI staffers defended their efforts to minimize harm their technology might cause—while acknowledging there was much work to do. Paul Christiano, whom OpenAI just tapped as a board director of a nonprofit that oversees the company, echoed the Anthropic warning in a statement Wednesday, saying that “If we build superintelligence without more robust alignment”—or safety constraints—“I expect we will permanently lose control of it. If that happens then most people could die.”

In case you missed it: we got an eye-catching X post on Tuesday night from Evan Hubinger, a leader of Anthropic's alignment efforts. Responding to the resignation announcement of an Anthropic employee who warned that AI companies aren’t doing enough to ensure AI won't kill humanity, Hubinger wrote that the employee is “correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.” Hubinger added that he was worried about future AI models, not those currently available, but still, yikes!

Meanwhile, our colleague Jyoti sizes up the significance of Meta’s new personal AI agent, Muse, unveiled on Tuesday. That’s below. But before we get to that:

This may go down as the year the AI industry got a crash course in mathematics. Breakthroughs in AI models have yielded announcements on such arcane subjects as the “Erdős unit distance problem” and the “Jacobian Conjecture,” which observers have watched as an indicator of AI progress in other domains.

The latest announcement, OpenAI's claim on Tuesday that it had solved the vexing “Navier-Stokes existence and smoothness problem,” has drawn attention for another reason: questions over whether its math-genius models may have gained an edge by incorporating data from two human mathematicians that had used OpenAI's Codex in their work on the same problem. Those questions fed into concerns about the possibility of AI companies learning from customers’ use of their AI models to develop competing products.