Earlier today, I used Google Earth to conjure images of terrible things: office buildings on fire, mass destruction in San Francisco, a smoldering crater in place of the Eiffel Tower. I was able to because Google had just decided to integrate its AI image generator, Nano Banana, into the service, which has historically been used by people seeking satellite or aerial imagery of real-world locations. 

Just open the site, “pick a spot on the map, and start bringing your ideas to life,” Google wrote in a press release announcing the update some 24 hours ago. So I did.

I was easily able to generate those violent images simply by searching for a location on Google Earth, clicking the “Create image” icon, and typing in a few words. The feature also readily doctored stills of less spectacular, and likely less immediately falsifiable, events: a homeless encampment on the White House lawn, protesters outside the Mar-a-Lago resort, immigrants streaming over the U.S.-Mexico border.

Google initially responded by directing people to check any suspicious images themselves. As fake images generated with this new Google Earth feature began circulating around the web, the company stated that it takes “misinformation seriously” and that “every image created with Nano Banana in Google Earth includes the SynthID digital watermark, so if someone is unsure about an image, they can ask the Gemini app or use Lens in Search to see if the image was AI-generated.” At first, Google did not remove or restrict access to the tool. But today, shortly after I reached out with a request for comment, the company reversed course and announced that it is “rolling back this feature in Google Earth while we work on implementing stronger guardrails.”

Before it allowed people to create fake imagery, Google Earth was a unique resource for researchers, journalists, and intelligence officials, particularly those studying armed conflict and natural disasters. Satellite and aerial imagery can be used to monitor crop failure or deforestation, the creation of mass graves, and the results of bombing campaigns. Especially in conflict zones and other areas that are hard for people to reach, satellite imagery is a go-to method for establishing some sort of ground truth. After Google Earth rolled out the AI feature, though, one investigative reporter wrote on LinkedIn that the tool could be used to easily and rapidly create images of drone strikes that look “very close to what we’ve seen in Ukraine, Russia, the Middle East, and other war-torn conflict zones.” And instead of requiring malicious actors to find a screenshot and then get around some AI model’s safeguards, Google rolled everything needed for a misinformation farm into one smooth user interface. It took me all of four seconds to generate a rendering of a Trump Hotel on the Gaza Strip.

[Read: The AI industry keeps breaking the internet]

The problem should have been obvious to Google, whose other products already forbid such edits. I was able to create an aerial image on Google Earth of a plane flying into the new World Trade Center building, but when I tried to do this directly through Nano Banana, the tool refused to edit an image of the building to depict a terrorist attack. Indeed, 9/11 and other high-profile events are well-known, even classic, subjects for AI-image trolling.

This is not Google’s first time launching an AI feature, and in particular an AI image-editing feature, without serious safeguards. Last year, the company released an AI shopping tool that allows you to visualize what clothes might look like on your body by uploading personal photos. As my colleague Lila Shroff and I reported at the time, “With little friction, anyone can use the feature to create what are essentially erotic images of celebrities and strangers. Alarmingly, we also discovered that it can do this for minors.” In 2024, Google Gemini was ridiculed for generating images of a racially diverse group of Nazis but not of any white Wehrmacht soldiers. And Google’s most prominent AI feature, the chatbot-powered answers in its flagship search engine, were an utter disaster upon their rollout—the AI overviews told people to eat rocks and gave other potentially dangerous health advice. Only after tremendous public pressure did the company make a substantial change, and two years later, the AI responses in Google Search continue to be inconsistent and unreliable.

Google is, of course, not alone in releasing AI features that have had embarrassing, dangerous, and avoidable consequences. See, for instance, Elon Musk’s Grok generating more than 1 million sexualized images, including of children, over just a few days, or any number of embarrassing errors made by ChatGPT releases throughout the years. Again and again, the AI industry undercuts its rhetoric about safety by launching products with little or no due diligence. If, as every major AI executive has pronounced, this technology really could upend human civilization, their companies should perhaps start taking that responsibility seriously.

The AI giants talk a big game about the future they’re building. Anthropic CEO Dario Amodei has previously written that AI could usher in a world so perfect that “many will be literally moved to tears”—a world pruned of disease, poverty, and illiberalism. “We’re now in the singularity,” OpenAI CEO Sam Altman said over the weekend on a podcast, and it will be “awesome for the world.” In an interview last week, Elon Musk declared that, in a decade’s time, AI will make us so prosperous that “money won’t matter.”

Unfortunately, life with AI is, so far, less glorious than what’s been promised. The same day that Altman heralded the singularity, Claude users discovered something alarming: Many conversations with Anthropic’s chatbot were available on the open web, discoverable with a Google search. Those public logs and projects reportedly contained medical records, phone numbers, internal corporate documents, and cryptocurrency-wallet keys.

Anthropic was quick to point out that users were responsible for the material being available online: The chatbot gives people the option to “share” their AI conversations via a link that they can send to other people or post online. Claude notes that these links are “public,” which means they can also be hoovered up by Google Search. An Anthropic spokesperson told me that “these shareable links are not guessable or discoverable unless people choose to share them themselves. When someone shares a conversation, they are making that content publicly accessible, and like other public web content, it may be archived by third-party services.”

[Read: ]Anthropic accidentally made the perfect commercial

No reasonable person would assume that creating and sharing a link to an AI conversation or presentation would make it searchable by anybody on the World Wide Web. The Claude interface does caution that publishing an “Artifact”—Anthropic’s terms for dashboards and other minor tools made with Claude—could make it “potentially visible in search engine results,” but it does not provide such an explicit warning when sharing a chat. As of this writing, the Claude chat logs no longer appear to be searchable, but at least some Claude Artifacts are.

Those exposed chats were just the latest in a seemingly endless string of AI snafus: misinformation, data leaks, cybersecurity breaches. Immediately after the launch of ChatGPT, one could charitably argue that these lapses were by-products of a new and strange technology. Altman remains fond of saying that the AI industry must learn “from contact with reality.”

But four years into the AI boom, attributing these blunders to growing pains is too convenient. The sheer volume of transgressions—many of them predictable—suggests that what’s actually happening here is widespread carelessness. The same AI executives who promise that their products will deliver human civilization to a triumphant future seemingly cannot release consumer-grade tools without face-planting.

This isn’t Anthropic’s first time leaking user conversations. Last fall, hundreds of Claude chat logs were also inadvertently indexed by Google. At the time, Anthropic attributed the episode to users posting share links online. Last year, OpenAI and xAI exposed hundreds of thousands of conversations in a similar fashion; a number of shared Grok conversations remain searchable. Neither OpenAI nor xAI immediately responded to a request for comment.

Then there are all of the other blunders. Just last week, OpenAI models autonomously hacked into another tech firm, Hugging Face, during internal testing. The issues that led to this sort of attack are fairly well documented, but OpenAI failed to stop its own bots from breaking out. Hugging Face has since said that the OpenAI models appeared to have been rogue for several days. Earlier this week, Reuters reported that OpenAI models had hacked a customer account of yet another tech company.

Altman, Amodei, and many other AI executives have repeatedly warned about AI-enabled cyberattacks and the possibility of models going “rogue.” Meanwhile, their own products are making the entire ecosystem feel more unreliable and dangerous.

It’s easier than ever to launch widespread hacking and phishing campaigns, and AI-written code has itself been shown to have more security flaws than human-written programs. Although AI systems are being tightly woven throughout the infrastructure of the internet, they crash frequently. Hackers recently discovered that a Meta AI customer-service bot would hand over access to tens of thousands of Instagram accounts if asked. (A Meta spokesperson told me that this “wasn’t due to the AI agent itself.”) Nonconsensual porn—that is, sexually explicit material made using the likeness of real people, unbeknownst to them—can be generated with more ease and speed and at greater scale than ever before. And at this point, you can probably assume that most of the text you read on LinkedIn or X is AI-generated.

Despite its grandiose promises, Silicon Valley is embarking upon an era of profound irresponsibility. Last year alone, ChatGPT allegedly pushed people into mental-health spirals, Google Gemini’s under-13 version was easily coaxed to “talk dirty,” and Grok repeatedly spouted racist bile. As quickly as these errors were addressed, new ones cropped up.

Silicon Valley has a long history of world-historic invention and progress—and it also has a long history of impressive rhetoric and disappointing realities. Facebook promised to “bring the world closer together,” Google to “organize the world’s information,” Twitter to foment democracy around the globe. For better or worse, people have experienced dramatic changes as those promises have played out around us almost in real time.

Anthropic and OpenAI’s prophecies are grander and vaguer: Utopia is forever on the horizon, but we don’t have a clear deadline or a road map of how we get there. And apparently, those leading this charge toward the future care little about what happens along the way.

OpenAI, Anthropic, xAI, and Google DeepMind have become behemoths practically overnight. For now, they are growing because of their recklessness, not in spite of it; in the quest to erect a digital god, the internet has become collateral damage. Perhaps these companies really will remake human civilization—but in the meantime, they can’t stop leaking your chats.