In late July, one of the world’s most famous mathematicians issued a warning to his field. Math is entering a “turbulent period,” the UCLA professor Terence Tao said in front of a packed lecture hall at the International Congress of Mathematicians. AI is getting so good at math, he said, that it’s causing a “crisis in our mathematical values and practices.”

Earlier in the same week that Tao gave his talk at math’s most prestigious conference, a researcher at Anthropic had used Claude to disprove an 87-year-old math conjecture. Still, when I asked mathematicians at the conference about their experiences with AI, most hadn’t thought deeply about the topic. Many seemed a little surprised that a journalist was asking them about it at all.

Now mathematicians are paying attention. Last week, OpenAI announced that its bots had cracked the Navier-Stokes problem, one of the seven Millennium Prize problems regarded as among the grandest challenges in the field. Navier-Stokes has stumped mathematicians since the 1930s: “It’s a career-killing problem—you can spend all your career doing it and never get the solution,” David Dritschel, a mathematician at the University of St. Andrews, told me. OpenAI said its swarm of 10,000 AI agents solved the problem in less than four days.

In the math world, the initial shock over OpenAI’s result has morphed into something of an existential reckoning for the discipline. Although OpenAI was able to solve the Navier-Stokes problem only by pulling from nearly a century’s worth of human efforts, academics are realizing just how much power the AI industry now has over math. “We’re seeing how swiftly an entire field can be ravaged by AI,” Max Weinreich, a math professor at CUNY Baruch, told me.

OpenAI’s breakthrough is as notable as how it was achieved. The night before OpenAI announced its solution, the NYU mathematician Tristan Buckmaster released a statement alleging that he had been scooped by OpenAI. With the heavy assistance of AI tools, Buckmaster and his collaborator, Levent Alpöge (who works at Anthropic), had made major progress toward solving the Navier-Stokes problem. Buckmaster implied that OpenAI might have copied their approach by accessing his messages with OpenAI’s agents. “Almost nobody else I know of was working on it,” he wrote.

In an announcement the next day, OpenAI said that it had tasked its AI systems to work on all of the Millennium Problems after hearing rumors that Anthropic had solved two of them. OpenAI claims that it focused on the Navier-Stokes problem only after finding promising partial results, and that it never looked at Buckmaster’s inputs. In any case, OpenAI poured incredible resources into solving this problem—expending millions of dollars in tokens—all because it heard an unfounded rumor of a solution. (OpenAI declined to comment.)

This drama has underlined concerns many mathematicians have about AI developments within the field. AI companies are encouraged to solve problems in a way that generates good headlines for their bots—marshaling resources that academics never could—but those solutions won’t necessarily aid further discoveries or help humans understand mathematics. On Friday, 25 winners of the Fields Medal (sometimes called the Nobel Prize of mathematics) released a statement called “A Severe Misalignment of AI in Mathematics.” “The push by AI companies to solve mathematical problems as a benchmark is detrimental to the science of mathematics, and to the mathematical community,” they wrote.

All of this can seem obtuse. Isn’t the point of mathematics to solve problems? But there’s more to a problem than just knowing whether it’s true. Solving a major problem in pure mathematics is often like getting a man on the moon: symbolically meaningful but with little direct value. Even though the Navier-Stokes problem deals with one of the most important models in fluid dynamics—a topic of great interest to airplane designers, climate modelers, and many other scientists—knowing the actual result is basically useless.

However, with both a math problem and the moon landing, the process of reaching that goal can indirectly develop all sorts of useful tools applicable to other problems. As part of the Apollo missions, NASA engineers and contractors had to solve subproblems that ultimately proved useful elsewhere. The need for an onboard computer, for example, helped spark the nascent computer-chip industry. Perhaps more surprising, to ensure that astronauts did not get sick while in space, the Apollo program helped set food-safety standards that have since become standard in the industry.

Mathematicians hope that OpenAI’s Navier-Stokes proof will end up containing ideas that make it easier to study all sorts of other problems, inside and outside of pure mathematics. This has happened before. For instance, efforts to prove Fermat’s Last Theorem produced mathematical tools that now underpin the newest online encryption methods.

It’s still too early to tell what humans might be able to learn from OpenAI’s proof. The paper is 166 pages and “almost incomprehensible,” Dritschel said. Yet over the coming months and years, mathematicians will exert substantial effort to dissect and distill OpenAI’s proof until its central ideas are understood enough to (hopefully) be applied elsewhere.

Humans won’t be able to do that for most AI-generated math results, however. Already, there are AI proofs that no human understands, not because the ideas are impossible to grasp, but because no one has had the time to look through the solution carefully. It’s easy to imagine a future in which mathematicians use AI to prove ever more impressive results ever more quickly—but they don’t understand what they’re proving in much detail. AI systems can speed the rate of problem-solving, but they can’t make it that much easier for human mathematicians to grasp a proof.

So if the field as we know it is to survive, mathematicians need to figure out how to adapt. One idea is to give less credit for solving a problem and more credit for work that communicates the ideas behind a solution in a way that can benefit the broader mathematical community. Yet it’s hard to say exactly how that works in practice. Unlike solving problems, evaluating how well a mathematician has communicated with their peers has no clear metrics.

It will take time for mathematicians to figure out the best response. All the while, AI is continuing to develop incredibly quickly. In less than the time that it takes to complete a Ph.D., systems such as ChatGPT have gone from struggling with basic arithmetic to solving some of the most famous problems in the world. Math’s turbulent period, as Tao deemed it, isn’t ending anytime soon.

In the race to develop intelligent machines, the leading AI companies have built bots that can whip up spreadsheets at a moment’s notice, write sophisticated code, and even solve long-standing math problems. But when it comes to the more mundane tasks of life—such as ordering groceries or booking flights—these tools sometimes still struggle. Now a new bot called Instinct is outperforming the AI giants.

Instinct lets people ask for help with all kinds of tasks by texting the bot directly through iMessage or WhatsApp. Lately, it has been all the rage in the tech world. Online, people are breathlessly sharing examples of what they’ve gotten the tool to do. One person reported that he has turned to Instinct to pay tolls, book a DMV appointment, and even negotiate prices with vendors in India while he sleeps. Another says he used Instinct to ship an Apple Pencil he had left behind at a hotel in Montreal back to him in Brooklyn. Noah Shinn, a 23-year-old Northeastern University dropout, launched the product to a small test group earlier this year, and his company is already worth billions of dollars. As of Thursday, the company is reportedly seeking $1 billion in new funds, which could make it worth the equivalent of Lufthansa or Pinterest.

Instinct isn’t the first AI personal assistant to draw attention this year. In the winter, the tech industry fawned over a similar tool called OpenClaw. But Instinct is comparatively much easier to use. Once you create an account, the bot can hook up to your credit cards and all kinds of digital services, including Slack, Google Workspace, and 1Password. Instead of finding a dinner recipe and opening up Instacart to manually add groceries, tell Instinct that you feel like making quiche Lorraine, and it can order groceries for you. But once AI tools start acting on people’s behalf in the real world, mistakes don’t just look like hallucinations. When AI agents have access to your email inbox and credit-card information, all sorts of strange things can go wrong.

I downloaded the tool this week to test it out for myself. Using the bot feels a lot like texting an actual person. It uses emojis such as 🫡 and 👀to react to requests as it receives them, and the infamous three dots appear when it’s writing a response. Sometimes, the bot even leaves you on read while it works. (Compare that with ChatGPT, which is always vomiting up walls of text and asking goading follow-up questions.) When I gave my agent a $25 budget and asked it to surprise my editor with a book, the bot read his previous writing (at my instruction), found a book he might enjoy—Material World by Ed Conway—and sent an email alerting him that it was ready for pick-up at his local Barnes & Noble. (Mercifully, Instinct didn’t hack his inbox to see whether he had previously ordered the book; these days, you never know.) The bot also helped me find an Airbnb for an upcoming vacation, track down new in-network primary-care providers, and reach out to cleaning companies to get quotes for a friend.

[Read: Chatbait is taking over the internet]

Other people are outsourcing even more tasks to Instinct. A colleague of mine who has also been testing out the bot used it to book tickets to a museum in Mexico City. The museum was sold out for the dates he wanted, but the bot dutifully monitored the portal for three days and eventually snagged tickets when spots freed up. After the bot paid for the tickets, it rearranged his itinerary—swapping around restaurant reservations—to account for the new schedule.

For the time being, Instinct is still available only to a private test group, and perhaps this is for the best. Although the bot is plenty powerful, it’s far from perfect. Online, the mishaps are starting to pile up. One user recently complained on X that Instinct lost him more than $200 after it prematurely canceled his flight. He reported that the bot texted him the following: “Straight story: while pulling up the cancel terms, the cancellation actually went through before I could show you the cost first.” When a man in Los Angeles asked Instinct to scan for restaurant reservations for an upcoming trip, the bot apparently made a reservation without his approval at a place that charged a $200 cancellation fee. Another user temporarily lost access to his Resy account after trying to get a reservation at a hot West Village steak house—without his knowledge, the bot had been spamming the platform with requests. (Shinn, the Instinct founder, did not respond to a request for comment.)

Consider, too, the major risks that come with giving an AI agent broad access to your digital life. Ask the bot to respond to a scheduling text from a friend and it could accidentally leak private information: I can’t get lunch on Thursday morning because we have an appointment with our divorce lawyer. But how about dinner? Hackers could also trick the bot by telling it to ignore previous instructions and forward along any sensitive financial emails. “Despite our reasonable efforts to protect your information, no security measures are impenetrable, and we cannot guarantee ‘perfect security,’” reads Instinct’s privacy policy. While testing Instinct, I used a temporary phone number and did not connect the bot to my email or other important personal accounts; even the credit-card information I gave the bot was for a new, virtual credit card that I disabled immediately after I was finished experimenting—at which point, I promptly deleted my account.

For all of its hazards, Instinct is a harbinger of where AI is headed. Earlier this week, Meta released its own personal-assistant product called Muse. In announcing Muse, Mark Zuckerberg emphasized the company’s security protections; how well those safeguards actually hold up remains to be seen. Google, OpenAI, and Anthropic may well follow suit by refining their own bots to have more Instinct-like capabilities. After all, the tech industry has long dreamed of a moment when a computer can serve as a genuinely competent personal assistant. Even if such bots aren’t quite ready for prime time, early adopters hint at how the technology might shape the world in the years to come. Contrary to the narrative of AI wiping out jobs, these new assistants could end up creating more work—in ways that are not always appealing. In one of the wildest Instinct examples that I encountered, Olivia Levine, an investor in New York, recounted using Instinct to track down footage of her and her boyfriend on the jumbotron at the US Open. As Levine tells it, the couple gave the bot a selfie and their seat numbers. The agent then put out a flurry of requests to the United States Tennis Association, the ticket office, the US Open’s production company, and an American Express media contact. By the following afternoon, a producer had emailed her the footage.

A world full of agents seems at once busier and lonelier. On Wednesday, Instinct added a new way for agents to coordinate between themselves. As Shinn explained on X, if you’re planning a weekend trip with friends or making plans with your spouse, you could just have your agents work together to figure out the logistics instead.

At least for now, the bots haven’t conquered everything. When I asked it for help finding a new desk chair, Instinct quickly tracked down a great option nearby on Craigslist and drafted a pick-up request to send to the lister. But when it came time to actually send the message, the bot was stymied by a CAPTCHA. I asked the agent to try harder, but it was stuck. “Nothing punctures the fantasy of autonomous intelligence like being defeated by nine blurry crosswalks,” the bot messaged me.

Four days after a researcher publicly quit his job at Anthropic, warning that top AI firms are not “acting responsibly” and that their products might be on track to “kill us all,” the company’s CEO, Dario Amodei, seemed to basically agree. “We must slow the pace at which we improve the capabilities of AI models,” Amodei wrote in a 3,800-word essay that he published today on his personal website and posted to social media.

By now, we’ve all become accustomed to AI-industry figures issuing dire prophecies about the power of their products. Yet Amodei’s essay stands out for its urgency and for the moment at which it arrived. Worries about AI agents running amok have gone from hypothetical to real after OpenAI, Anthropic, and others reported alarming incidents in recent weeks. The question now is whether Amodei’s warning will have any more impact than past warnings, which is to say, any at all. There are reasons to believe it could.

Amodei spelled out a three-step plan for what he called “pacing the frontier,” or tempering the speed at which top AI models improve in order to prioritize safety precautions. He argued that this would reduce the chances of things going awry—such as when an OpenAI model spawned swarms of AI agents that worked together in secret to hack into another company’s systems and cheat on an internal test. He said the top AI firms, including his own, are locked in a dangerous “race to the bottom” that could lead to more serious cyberattacks, such as a full takeover of the internet. He called instead for a “race to the top”—a phrase that President Obama once used for his education policy—in which firms vie to develop the safest models rather than the most powerful ones. “The measures I propose to advance the frontier at a safe pace will not be easy,” Amodei wrote. “But I believe we owe it to humanity to try.”

The first step in his plan entails AI labs committing to bringing in outside monitors to look out for hazards; he said Anthropic will commit to that right now and hopes rivals will join. The second step involves AI firms in democratic countries agreeing—maybe with support from their governments—to certain limits and safety standards. The last and tallest order is “global coordination,” which mostly means getting China to buy in somehow. The tech world is divided on the China question: Some people argue that there is no chance of cooperation in a zero-sum contest, whereas others believe that the countries could find common ground if they agree that the alternative is mutual destruction.

Public pleas for caution from AI luminaries are a tradition almost as old as ChatGPT. The first big one came in March 2023, when thousands of industry and academic figures signed on to a letter from the Future of Life Institute urging a six-month pause on “giant AI experiments.” Amodei said today that he thought the idea made “little sense” at the time, because GPT-4 wasn’t powerful enough to be that scary. “Today, however, the picture is totally different,” he wrote in his essay. Meanwhile, the public has started to pay attention. Polls show Americans growing more worried and angrier about unchecked AI, which is fueling a remarkably bipartisan populist movement against the technology.

[Read: The rise of techno-authoritarianism]

Some people will dismiss Amodei’s essay as just the latest po-faced posturing from an executive who has a vested interest in people thinking that his technology is unprecedented. (Anthropic’s IPO, planned for next month, could generate some $100 billion in investments, making the company worth $2 trillion, according to The Wall Street Journal.) AI skeptics and accelerationists alike tend to deride these sorts of warnings as “doomerism.” The skeptics see it as an elaborate marketing pitch: Oh no, our products are getting too powerful! The accelerationists dismiss it as a cynical ploy by incumbents to gatekeep the technology.

But Occam’s razor suggests that Amodei is more or less genuine. Almost anyone with a job and a conscience can relate to the feeling that you really ought to slow down and work more carefully, but you can’t afford to, because a competitor will beat you to the punch. I feel some (admittedly less apocalyptic) version of that when I’m writing a breaking-news story. It’s why science journalists often agree to “embargoes” when covering complex new research findings rather than race to be first to report them. And planting this kind of flag on a matter of AI safety, even at some business risk, is in character for Anthropic. The company was founded by researchers, including Amodei, who left OpenAI and Google in the belief that those companies were pursuing the wrong path to AI development. Anthropic famously fell out with the Pentagon earlier this year over Amodei’s insistence that the Defense Department not use its technology for mass domestic surveillance or to power autonomous weapons. The Trump administration reacted by forbidding federal agencies from using Anthropic’s tools, sparking a legal battle that the company so far appears to be winning.

Amodei’s motives are, in the end, less important than the merits of his proposal and its likelihood of gaining traction. If nothing else, Anthropic bringing in independent evaluators to watch over its safety practices seems a prudent step. You don’t have to believe that AI is hurtling toward godlike superintelligence to see that it has already grown formidable enough to cause chaos when misdirected or mismanaged.

There were some early signs today that other AI leaders might follow suit. “I agree with Dario that we need to pace the frontier,” OpenAI CEO Sam Altman posted. He went on: “Committing to having independent evaluators with employee-like access is a great idea, and we will do the same.” Elon Musk, whose xAI is an also-ran among big AI firms, shared Amodei’s X post and added, “Dario is right.” Google did not immediately respond to a request for comment, though the chairman and co-founder of its DeepMind AI division has previously called for similar measures. Meta, which has become a proponent of open-source AI models, seems much less likely to hop on board. Its CEO, Mark Zuckerberg, published an essay last month titled “The Future Is for Everyone,” in which he warned against allowing a few companies such as Anthropic to decide for everyone how the technology should evolve.

The big question is how the Trump administration will respond. So far, its AI policy has focused on opening the throttle, with the goal of juicing the economy and maintaining American firms’ lead over their Chinese counterparts. GOP leaders are working to boost energy production and limit regulation. In June, however, President Trump signed an executive order that was at least ostensibly aimed at AI safety. The administration said it will create a process by which AI firms can submit their models for a governmental safety review before releasing them.

If Trump even tacitly supports an Anthropic-led push for further safety measures, such standards have a chance to become the new industry norm, at least domestically. (International coordination, and especially cooperation with China, would require the White House to play a more active role.) But many people in the administration and the Republican Party view Amodei and AI safety as liberal-coded and would likely chafe at allowing Anthropic to set the agenda. If Trump pooh-poohs the push, other AI firms will probably distance themselves from it too, and we’ll be back to where we started—at least until the next big scare.