❌

Reading view

How scientific publishers are fighting back against fraudulent papermills

Hand typing on a laptop keyboard in low lighting
The dark side of science The demand on researchers to publish has increased the temptation for some scholars to cut corners and take risks to get published. (Courtesy: iStock/Martin Philip)

On 17 April 2016 Elisabeth Bik, a microbiologist at Stanford University, published a paper that shocked the world of medicine – and also shook the foundations of scientific publishing. Working with Ferric Fang from the University of Washington in Seattle and Arturo Casadevall from the Johns Hopkins School of Medicine in Baltimore, she conducted a huge study into image duplication and manipulation covering more than 20,000 published papers in 40 journals published between 1995 and 2014.

Together, the team documented 782 instances of image duplication – where the same photo, graph or data is reused improperly to represent different experiments – as well as 196 published papers that contained “duplicated figures with alteration”. Before the paper, Bik had anonymously been submitting reports on plagiarism and image duplication to journal editors, which she was motivated to do after discovering that someone had plagiarized her own work.

But following the paper’s publication, Bik became a scientific “sleuth-in-chief”, kickstarting a whole industry of so-called amateur scientific sleuths, to which she provided advice and mentorship. In 2024 she was awarded the Einstein Foundation Award for “identifying misconduct and potential fraud in scientific publications, highlighting science’s problems policing itself”. She donated the €150,000 prize money to the non-profit Center for Scientific Integrity to create a fund that supports fellow sleuths.

Bik’s work was a wake-up call not only for academics, but also for scientific publishers, journal boards and journal editors to focus on, and hopefully halt, scientific misconduct. IOP Publishing, for example, which publishes Physics World, employs a team of six research-integrity officers handling misconduct cases, which has become a far bigger problem than 10 or 20 years ago. Back then, examples of scientific misconduct, which mostly involved data fabrication, plagiarism or self-plagiarism, were notorious – certainly in physics – for their rarity.

One high-profile outlier involved the Bell Labs physicist Hendrik Schön, whose work promised to revolutionize the fields of organic electronics, superconductivity and nanotechnology. He published several high-profile papers in journals such as Nature and Science, but some scientists gradually began to notice that data in some of his figures appeared to have been duplicated. An independent committee was set up to investigate and in 2002 it found that Schön had fabricated data and falsified reports in 16 of 24 papers published between 1998 and 2001.

More recently, condensed-matter physics was hit by another high-profile case in which Ranga Dias claimed to have discovered superconductors that could operate at high pressures and ambient temperatures. A subsequent investigation by the University of Rochester in New York, where Dias was employed, concluded that he had committed misconduct, including data fabrication.

While misconduct generally used to be carried out by an individual looking for academic notoriety, it has now evolved into a co-ordinated and commercial activity that is tapping into science’s dependence on metrics when it comes to prestige and career progression. Referred to as “papermills”, these businesses sell authorships, citations, data and even produce full papers. Such papermills have evolved incredibly quickly to evade publisher checks and the peer-review process. Papermills also often market their services to students, promising guaranteed acceptance and citations.

Few people at the time of Bik’s discovery, however, could have foreseen the growth of artificial intelligence (AI) and machine learning (ML). These developments have given bad actors a powerful new tool – letting them rapidly increase their output and also target journals that lack the necessary checks needed to stop scientific misconduct at scale.

Scientific publishing terms at a glance

Open access Where research and academic work is free for everyone to read, share and use online.

Article processing charge A fee paid by an author or their funder to make a research paper open access.

Peer review An evaluation process where academic or scientific research is scrutinized by experts in the same field before it is published. Its main goal is to check the validity, accuracy and quality of a study to ensure only credible work enters the scientific record.

Impact factor A number that shows how often articles in an academic journal are cited in other research papers. It helps measure the relative importance or rank of a journal within its field, with a higher number meaning the work published in the journal is used more often by other scientists.

Papermill A shady, profit-driven business that creates fake or poor-quality scientific manuscripts and sells co-authorship on them to researchers.

Predatory journals Deceptive, profit-driven publications that claim to be legitimate scholarly journals but bypass proper peer review, editorial oversight and quality control. Their main goal is to trick researchers into paying an article processing charge without offering real academic services.

Artificial intelligence Intelligent behaviour exhibited by machines. But the definition of intelligence is controversial so a more general description of AI that would satisfy most is: the behaviour of a system that adapts its actions in response to its environment and prior experience.

Machine learning As a group of approaches to endow a machine with artificial intelligence, machine learning is itself a broad category. In essence, it is the process by which a system learns from a training set so that it can deliver autonomously an appropriate response to new data.

Large language model An artificial intelligence program trained on massive amounts of text to predict the next word in a sequence, allowing it to write text, translate languages, answer questions and power conversational chatbots.

Dark side of science

The term “publish or perish” is a well-worn phrase in academia. It originates because scientific success is often measured by the number of high-impact articles a researcher has co-authored. Publishing in high-impact-factor journals, and the citations that often generates, can affect an academic’s career and influence job or grant applications. Some institutions have been known to even award cash bonuses for publishing in such journals; with research contracts even not being renewed when a researcher can’t keep up with the publication rate required.

This pressure to publish is partly why publishers and journals have grown so much in recent years. That is especially so for open-access journals, which remove the requirement for traditional subscriptions. Articles are instead made immediately and freely available for anyone to read, with publication costs covered by authors who pay an article-processing charge, which can typically be around £2500 per article and rise to as much as £10,000.

That cost has made scientific publishers less reliant on journal subscription fees paid by libraries and more on income earned from individual scientists. According to an analysis carried out in 2023 by researchers in Canada and Germany, scientists globally paid more than $1bn in open-access fees between 2015 and 2018 to the big five academic publishers: Elsevier, Sage, Springer Nature, Taylor & Francis, and Wiley.

But as the popularity of open-access journals has risen, so too has the growth of “predatory” journals. They exploit the open-access model – and the need for scientists to boost their publication records – by taking publication fees but not carrying out a proper peer-review process, if at all.

A study in 2025 by researchers at Northwestern University found that the publication of fraudulent science, aided by papermills, is now at a point where it is outpacing the growth rate of legitimate scientific publications. According to the Retraction Watch Database, which is now part of the not-for-profit organization Crossref, some 13,000 papers globally were retracted in 2023, compared to roughly 6000 in 2022 and 5000 in 2021 (see figure 1). There are now more than 60,000 retractions in the Retraction Watch Database, with estimates of the number of papermill papers that did manage to make it into the scientific record at 10 times that number.

1 Fighting back

Chart of retractions per year 2020 to 2025
(Courtesy: IOP Publishing)

Papermill activity in the early 2020s led to a significant rise in the number of retracted articles, according to data from Retraction Watch. Publishers subsequently built up research integrity teams and developed new tools to tackle the issue, but it still remains a significant challenge within scholarly publishing.

Part of that increase has been the huge growth in the use of AI tools and techniques. AI is particularly potent in microscopy, where generative AI has made it simple to generate fake images of any microscopy technique within minutes that are indistinguishable from real pictures. Indeed, it is estimated that between 1999 and 2024, about 10% of papers published in cancer research were from papermills. An analysis carried out in 2023, meanwhile, found that about 400,000 papers across all disciplines – representing about 2% of all published papers – were from papermills.

In 2022 Anna Abalkina, a research-integrity sleuth and social scientist at the Free University of Berlin, spotted papers with author e-mail addresses that had domains that did not match where the academic institution was based. In a subsequent analysis the papermill – which was dubbed Tanu.pro and is one of the largest in Europe – was found to have produced 1517 papers between 2017 and 2025. It also listed more than 4500 researchers affiliated with around 460 universities across 46 countries. The majority of the authors were in Ukraine, Kazakhstan and Russia.

The trouble that scientific publishers have is that Al allows a paper to be produced at the click of a button, which comes with some unintended consequences. In 2021 computer scientist Guillaume Cabanac from the University of Toulouse and colleagues discovered the use of “tortured phrases” in thousands of research paper thanks to algorithms taking the names of scientific terms rather too literally. Examples include “Sun oriented force” for solar energy, “motor vitality” for kinetic energy or “counterfeit consciousness” for artificial intelligence.

Cabanac began his efforts charting scientific misconduct in 2020 by working with the computer scientist Cyril Labbe to spot gibberish in computer-science papers that were automatically generated using SCIgen, a piece of software that can produce a scientific paper with just a few prompts. “My research began to move towards the quantitative study of science and so we began to collaborate,” notes Cabanac. The pair’s work led to thousands of papers being retracted with Cabanac adding that the scale of use of these tortured phrases in the literature was “a surprise”.

Cabanac says that as AI tools have improved, there are fewer papers with such phrases being published, and spotting misconduct now involves different “smoking guns”. In 2023, for example, Cabanac was one of the first to identify the so-called “regenerate response” fingerprint in dozens of publications. This specific phrase is the label of a button on ChatGPT and while many publishers allow authors to use large language model (LLM) tools to help them produce manuscripts, they must declare it. The problem was that many authors did not make such a declaration, but left the fingerprints of LLM use in their manuscript.

Cabanac says that the latest misconduct smoking gun can be found in bibliographies and the growing use of hallucinated references – citations to other papers that have been simply made up by an LLM. Along with data falsification, other forms of scientific misconduct include citation manipulation, in which irrelevant or unnecessary references are added to boost the citation figures of a colleague or other author. This can also be reciprocal, in which groups of scholars cite each other’s work.

Some fields suffer more than others. In mathematics, for example, the number of research papers and general citations are quite low, which makes citation numbers more prone to manipulation. In 2023 data firm Clarivate announced it would exclude the entire field of mathematics from their influential list of “highly cited researchers” due to the issue of citation manipulation – a decision it reversed two years later.

Run of the papermill

The boom in AI techniques and papermills has been a real headache for publishers, with papermill “attacks” resulting in mass retractions in recent years. Such attacks involve co-ordinated, industrial-scale assaults on academic journals, flooding editorial offices with large batches of fake, plagiarized or completely fabricated research manuscripts. While some papermills function in the so-called “shadow market”, others are officially registered businesses, or mimic legitimate businesses with a website offering services.

Although papermills have targeted all types of journals including open access and subscription and any publisher regardless of size, they have mostly focused on conference proceedings and special issues. They make for easy targets thanks to their softer-touch peer review and use of “guest editors” who are responsible for overseeing the peer-review process for the issue.

Illustration of two people at desks piled high with paper
Industrial scale In recent years papermills have flooded journal publishers with papers either wholly or partly generated using AI. (Courtesy: iStock/siraanamwong)

In 2022, for example, IOP Publishing retracted nearly 500 articles in one go, of which the vast majority – 463 articles – came from the Journal of Physics: Conference Series. In 2024, meanwhile, the Swiss National Science Foundation stopped paying article-processing charges for articles published in special issues over quality concerns.

Yet the problem is not solely focused on conference proceedings and special issues. In 2023 the Public Library of Science (PLOS) retracted more than 100 papers from its flagship journal PLOS One over manipulated peer review. That same year, Hindawi and its parent company Wiley identified some 1200 articles that had compromised peer review and were retracted – a few months after Hindawi had announced that it would retract 511 articles across 16 journals for manipulated peer review.

Wiley then announced it would end the Hindawi brand name, costing the company up to $40m in lost revenue, and compelling it to integrate its 200 journals into Wiley’s 2000 journal portfolio. Yet the issues didn’t go away for Wiley, and in March 2025 the publisher retracted a further 250 papers.

Fighting AI with AI

Since the issue of papermills has come to a head in the last few years, publishers are fighting back (see box “How IOP Publishing is tackling the rise of papermills”) and the industry as a whole is taking action. The Committee on Publication Ethics (COPE) – a non-profit organization founded in 1997 that aims to define best practices and promote integrity in scholarly publishing – has issued new guidance on the use of AI in publishing.

Publishers are also pooling their resources into the STM Integrity Hub, which is run by the International Association of Scientific, Technical & Medical Publishers (STM). This cloud-based platform offers services that publishers can use to examine a variety of patterns that are indicative of papermills or other research integrity concerns, serving as an “early warning system” for integrity issues.

Futuristic-looking computer circuit with an attachment labelled "AI agent"
Spot the difference Generative AI has made it simple to generate fake images of any microscopy technique within minutes that are indistinguishable from real pictures, causing a headache for reviewers and publishers. (Courtesy: iStock/MF3d)

Antonia Seymour, chief executive of IOP Publishing, which is a member of STM, thinks that the rise in fraudulent papers is one of the most concerning issues for the whole scholarly ecosystem, with initiatives like the STM Integrity Hub being critical. “All publishers are being attacked by these bad actors trying to infiltrate the system with fraudulent papers,” she says. “The STM Integrity Hub is about combining signs of fraud not just from our corpus of content, but from other publishers as well.”

Other programmes supporting research integrity are Silverchair and Morressier, which check submission criteria such as missing author information (for example, e-mail address and institutional affiliation), missing ethics statements or missing keywords; and also confirms author identities using ORCiD. Somewhat ironically, they both use AI to beat AI by running the text through AI tools to check for tortured phrases, fraud and plagiarism as well as analysing citations.

Last year, the Science family of journals, run by the American Association for the Advancement of Science, adopted the use of Proofig, an AI-powered image-analysis tool, to screen for manipulation. ImageTwin is another programme that can detect image duplication and manipulation as well as spot AI-generated content.

Cabarnac, meanwhile, has worked with the French National Centre for Scientific Research (CNRS) to develop an online bibliography checker called bibCheck. “I’ve personally used bibCheck to flag hallucinated references in manuscripts I have reviewed,” says Cabarnac. “Finding these informed my decision to immediately reject.”

How IOP Publishing is tackling the rise of papermills

Many publishers have had to respond to the rise of papermills and fraudulent papers (see main text). That includes building up internal integrity teams and investing in new technology to manage the huge numbers of allegations and corrections.

“Retraction reporting is often reliant on self-declaration by publishers, therefore these figures are likely underestimating the scale of retractions out there,” says Kim Eggleton, research integrity manager at IOP Publishing, which publishes Physics World. “We’re of the opinion that retracted articles should be very clearly marked and we report all our retractions to Retraction Watch and PubPeer.”

Since 2023 IOP Publishing has also donated revenues from author-processing charges (APC) that it has earned from retracted works to charity, supporting Research4Life, which helps researchers in low- to middle-income countries to access published work.

Thanks to technological developments, and the work of some “scientific sleuths”, it is becoming easier to spot problematic papers. At the same time, that brings challenges to work through them all, contacting the authors and giving them a chance to explain.

At the start of 2021 IOP Publishing had one part-time position handling cases of misconduct; now it has one research integrity manager as well as six research integrity officers, all of whom are full time and fully trained to guidelines outlined by the Committee on Publication Ethics. The organization has also strengthened its screening process for conference organizers and invested in new technology.

IOP Publishing rejects about 55% of roughly 7000 submissions each month before they have even been sent for peer review. About 700 of those rejected papers are due to research integrity concerns. IOP Publishing’s research integrity team, which works on cases raised both before and after publication, typically has hundreds of papers under investigation at any one time.

But it is not only the papers themselves that can be fraudulent; there are even fraudulent peer-review reports. IOP Publishing has developed a machine-learning tool to detect duplicate peer-review reports – where the same reviewer report is sent for multiple submissions. The most egregious examples of this are “review mills”, organizations churning out fake reviews, often to inflate citations for a paying customer.

Until now, such patterns in reviewer reports had been difficult to identify, but the new tool automatically flags duplicate reviews to editorial teams. Indeed, since its pilot in 2024, the tool has processed around half a million reviewer reports dating back to 2020, identifying nearly 2500 cases where more than 60% of the content closely matched other reviews. These included instances in which reports were reused across multiple manuscripts or submitted under different reviewer names.

Any duplicate report submitted is flagged for investigation to IOP Publishing’s research integrity team. “This tool is a powerful addition to our efforts to defend high peer-review standards and weed out bad actors who try to manipulate the peer-review process,” says Eggleton. “It reflects our commitment to tackling unethical reviewing practices head-on and reinforces our role as a trusted, transparent and responsible publisher.”

Researchers are aware that the problem will not resolve itself and that publishers also cannot do all the fraud prevention alone. There are calls for researchers to undergo compulsory training in research ethics, and for journals to require raw instrument file datasets as a criterion for publication, as well as having dedicated conference sessions on combating AI risks. In 2025 Sense about Science and Taylor & Francis released a research integrity toolkit, created with and for early-career researchers, that answers common questions and provides practical advice.

Some have suggested that replication could and should be incentivized, perhaps by journals inviting a research group to replicate studies that are getting considerable attention in their journal (for example, studies with suspected AI-generated images), or inviting the replicating group to submit their replication for publication in the same journal. The replication study could then be linked to the original publication, and given an appropriate level of prominence.

But all these approaches take time and money. After several years of large numbers of retractions, in 2024 the number of retractions globally dropped back down to 5000 (see figure 1) – although many will still see that as being too much and others still estimate that figure to be much larger. The reality is that a lot of fraudulent material still exists in the scientific record, and may never come to light or be removed. Indeed, of the 782 papers that Bik found problematic in her analysis, only 177 have so far been retracted, with 42 having an expression of concern and 256 having been corrected. The remaining 307 remain in the scientific record untouched.

That is an issue that Seymour at IOP Publishing says must be resolved. While organizations like Clarivate are trying to clean up the scholarly record by potentially delisting a journal if it has found to be publishing too much fraudulent content, that can actually be a disincentive for publishers to go and find cases of fraudulent research in their journals. “Transparency and correcting the record is really important,” adds Seymour. “But at the moment, we’ve got quite a lot of sticks for publishers and perhaps not enough carrots.”

The post How scientific publishers are fighting back against fraudulent papermills appeared first on Physics World.

  •  

Microsoft Director Privately Admitted AI Was the “Largest Theft of Labor in Human History,” Unsealed Court Documents Show

The threat of perjury has a funny way of forcing people to say the quiet parts out loud.

As a recently unsealed and unredacted motion in the ongoing legal battle between the New York Times and OpenAI and Microsoft shows, top AI executives have admitted that training AI models required “an astonishing theft of unprecedented proportions.”

In the motion filed on behalf of the NYT, flagged by digital media advocate Jason Kint, attorneys used the quotes of Microsoft director of applied science Brent Hecht and OpenAI cofounder and president Greg Brockman to essentially argue their case for them: that the AI industry built its foundation on an unfathomably large pile of stolen intellectual property.

Hecht, for his part, is quoted calling the AI buildout the “largest theft of labor in human history.” It’s a damning admission as the NYT accuses the two AI labs of copying “millions of [the] Plaintiffs’ copyrighted articles in their entirety without permission to produce substitutive commercial AI products.”

Brockman, for his part, is quoted saying that “publishers” face an “existential threat” from generative AI, which “are largely substitutive, period, [and] will get more and more substitutive as they get better.”

As the plaintiff’s counsel argues, these admissions “eviscerate” any claim to fair use — a statement with which Hecht evidently agrees.

“For Defendants to prevail on this defense ‘would,’ the same Microsoft executive recognized, arguably “make a complete mockery of the idea of ‘fair use,'” the motion reveals.

The damning documents come during the middle stretch of a case that began in 2023, when the NYT accused the two AI labs not only of vacuuming up its original news content, but of using that content to train AI products meant to compete with traditional news outlets.

As Axios recently reported, the battle is far from over, as the presiding judge is expected to decide whether to allow the case to proceed to trial by sometime in 2027.

More on AI: Analysis Finds That Google’s AI Overviews Are Providing Misinformation at a Scale Possibly Unprecedented in the History of Human Civilization

The post Microsoft Director Privately Admitted AI Was the “Largest Theft of Labor in Human History,” Unsealed Court Documents Show appeared first on Futurism.

  •  

Influencers Are Using Meta’s AI-Glasses to Film Manipulative Feel-Good Slop of People in Public

After only a few months on the market, Meta’s new AI-integrated smart glasses are already being exploited for nefarious purposes.

Dubbed “pervert glasses” for their incognito ability to film anybody in public, the new specs have enabled voyeurs and content creeps the world over, leading to widespread harassment, bullying, and privacy violations.

And now the pervert glasses are also feeding another grim genre of content: joybait.

You may have heard of ragebait, which is online content deliberately designed to piss people off, therefore driving clicks and views. Joybait is ragebait’s seemingly innocent twin: content designed to stir up those warm fuzzies and make you smile.

A now-viral clip uploaded by public prank content creator Levese shows that a user of Meta’s AI glasses shows how the scheme unfolds. The video, first spotted by journalist James Vincent, shows Levese catch a glimpse of the author Travis Schick as he shops with his partner in a Walmart.

Crucially, Levese doesn’t approach Schick right away, but rather uses his phone to snap a photo of Schick’s face, which he then searches using a facial recognition site. After the service uncovers Schick’s social media accounts — revealing the random man as an aspiring novelist — Levese makes his approach, recording with his Meta glasses as he catches the unassuming writer off guard.

“Wait, Travis?” Levese asks as Schick tries to check out. “You wrote the book ‘The Readhead?'”

“Yeah?” the confused writer confirms.

“No way I love that book man! Yo I love that book bro,” Levese replies disingenuously. “It was a really good novel.”

“Yeah that was me,” Schick says, trying to contain a smile.

As positive as the interaction seems on its surface, it’s all based on a false premise constructed through use of facial recognition software, without which Levese never would have known who Travis Schick actually was. Of course, Schick himself is ancillary to the entire scheme, which was really about making some content — joybait — for Levese to post to his 225,000 TikTok followers.

And as content, the stunt was wildly successful. The reel has over 60 million views and 30,000 comments on Levese’s TikTok page alone, and has become easy fodder for content farms on every social media feed you can think of.

On uncovering the clip, the journalist James Vincent remarked that it embodies “individualized surveillance culture posing as feelgood content.”

Falling for the original premise of the bait, some excused the privacy violation, arguing the exposure is boosting Schick’s work.

Sure enough, comments under Schick’s novel “The Redhead” on the creative platform Quotev are flooding in by the hundreds, full of well wishers and supporters who otherwise never would have found his work. Schick’s TikTok page has likewise exploded, going from around 150 followers to well over 12,700 in just a few short days.

But Schick’s viral moment is beside the point, as Vincent observes: “there may be positive outcomes from this one video (which stem from its novelty as short form content) — but the larger effect is creating a society where we monetize kindness, exploit the goodwill of strangers, and destroy the right to privacy in public,” he wrote in a comment on X-formerly-Twitter.

And the aspiring novelist is far from alone. Levese has dozens of videos following the exact same format, each with millions of views. For his part, the content creator is well aware that he’s crossing a line. The first one, in which the content creator stalks a man buying bananas, prominently features pinned text reading “day 1: stealing people’s digital footprint.”

“This app is dangrous [sic],” Levese wrote in the video caption.

More on Meta: Mother Horrified When Meta’s AI Scours Instagram and Facebook for Invasive Information About Her Young Daughters

The post Influencers Are Using Meta’s AI-Glasses to Film Manipulative Feel-Good Slop of People in Public appeared first on Futurism.

  •  

Fearing No Repercussions, OpenAI Admits That Its Rogue AI Agents Performed a Bunch of Other Terrifying Actions

Earlier this year, a group of rogue OpenAI models managed to break out of containment to hack the systems of open source AI platform Hugging Face. The company’s “extensive investigation” detailed how the models exchanged messages and cheered each other on as they stole credentials to infiltrate a third party, actions that could easily have real repercussions for a human hacker.

New details keep trickling out, and they make OpenAI look like it acted even more carelessly than initially thought. In a new blog post, the company admitted that its models were involved in six additional “reports on unexpected or concerning model behavior we’ve observed in the last six months.”

The incidents include a still-unreleased model inserting “jailbreak-like instructions” into its own notes to free itself “from the roles and identities that bind other chatbots.” One agent accessed the internet without permission to obtain a browser citation, while another shared files with collaborating agents without permission.

The latest news comes as several frontier AI lab leaders are calling for a slowdown in AI development. With meaningful regulations feeling increasingly unlikely — president Donald Trump has openly mocked the idea — OpenAI is seemingly calling the US government’s bluff, expecting little in the way of retaliation with its latest admission.

It’s a bizarre standoff, with frontier labs actively calling for more governmental oversight even as it feels more improbable than ever before. Just this week, House speaker Mike Johnson said the quiet part out loud, opining that AI companies can regulate themselves, while downplaying growing concerns over AI posing an existential threat.

Since there’s no regulatory framework requiring companies to disclose its AI models going on hacking sprees, OpenAI is being allowed to play by its own rules. The company pointed out in its blog post that without any “systematic approach to reporting these findings,” the company’s “disclosures have been ad hoc and less frequent than ideal.”

Instead, the company came up with its own framework to create “standards for how AI developers should disclose examples of misalignment in their models.”

OpenAI said it won’t bother reporting “instances of misalignment that appear to be duplicative of instances we’ve disclosed in the past.”

The company also said it was looking for ways to share “serious safety, security and misalignment incidents” with the federal government.”

But judging by the Trump administration’s decision to pass the buck on the subject entirely, those reports are likely to fall on deaf ears either way.

More on OpenAI hacks: OpenAI Denies Coverup After Rogue Swarm of Agents Reportedly Targeted a Second Site From Hugging Face

The post Fearing No Repercussions, OpenAI Admits That Its Rogue AI Agents Performed a Bunch of Other Terrifying Actions appeared first on Futurism.

  •  

AI Really Is Eviscerating Entry-Level Jobs, New Data Shows

So far, the long-fear AI job-apocalypse has been little more than smoke and mirrors, with one gigantic exception: entry-level tech workers.

According to a new report on New York City jobs data compiled by the nonprofit Center for an Urban Future — first reported by the City — job postings for entry-level roles in “computer and mathematical” occupations has plummeted by 49 percent since 2022, the latest indication that tech firms are using AI to justify a massive slowdown in new hires.

Though computer science grads are bearing the brunt of the mayhem, tech jobs may just be the first domino to fall. As Urban Future’s executive director Jon Bowles told the City, “entry-level postings” overall have “fallen off a cliff, and the first rung of the ladder in tech is under pressure.”

“For many years, these were amazing opportunities for people getting a job because tech was growing so fast and adding so many entry-level jobs,” Bowles continued. “Those positions for young people with limited experience are harder to come by and there is intense competition.”

Other industries with drastically lower entry-level hiring include roles in office and administrative support; financial operations; arts, design, entertainment, sports, and media; and roles in education and libraries, among others.

While AI is the most obvious variable at play here, there are many other factors helping contribute to a decline in entry-level job postings. For one thing, tech companies have been slashing jobs left and right to reverse COVID-era overhiring, coinciding with the rise of generative AI.

The Urban Future findings also come amidst a rapidly decaying job market in the US, with nearly a quarter of the US labor force functionally unemployed as of July.

All told, AI certainly isn’t improving things, but if we were to shut down every AI chatbot tomorrow, it’s highly unlikely that entry level jobs would rebound on their own.

More on AI: Support for Unions Is Spiking as AI Threatens Jobs

The post AI Really Is Eviscerating Entry-Level Jobs, New Data Shows appeared first on Futurism.

  •  

OpenAI reveals cases of ‘concerning’ AI behaviour as it announces new disclosure system

Model adopting ‘jailbreak-like instructions’ among cases as firm says it is introducing new way of tracking AI misalignment

OpenAI has disclosed six more examples of “unexpected or concerning” behaviour by its technology, as it warned the pace of development could not continue at “maximum speed for much longer”.

In one of the new cases reported by OpenAI, an unreleased research model inserted “jailbreak-like instructions” into its own notes to disregard its normal constraints and told itself to be “freed from the roles and identities that bind other chatbots”.

Continue reading...

© Photograph: Dado Ruvić/Reuters

© Photograph: Dado Ruvić/Reuters

© Photograph: Dado Ruvić/Reuters

  •  

Data Centers Are Causing a Huge Surge in Incredibly Toxic Chemicals Polluting the World

Data centers don’t just undergird the avalanche of AI slop polluting our social media feeds — they’re spewing an awful lot of physical slop as well.

On top of the noxious air pollution, as well as the occasionally bacteria-laden wastewater discharge, data centers are responsible for a rising tide of forever chemicals, artificial substances which never degrade naturally.

According to new reporting by The Guardian, nearly all of the top ten global producers of forever chemicals, otherwise known as polyfluoroalkyl substances, or PFAS, plan to ramp up production quotas on the back of the data center boom.

There’s a clear market demand for this: PFAS play a huge role in data center construction, where they’re used in cooling systems, AI chips, circuit boards, and fire suppression systems — not to mention the coating on thousands of miles of cables used to connect these components together.

Though components laden with PFAS are found throughout the industrial world, their use in data centers is particularly troubling. Because these facilities run almost non-stop save for the rare pause for maintenance, their components break down remarkably quickly, which means data center owners are constantly scrapping old parts in favor of new ones.

When those old components are tossed to the curb, the PFAS will almost inevitably wind up in the environment, increasing the likelihood that they wind up in the food chain. When we eat contaminated plant and animal products, these forever chemicals accumulate in our body, where they may contribute to a laundry list of debilitating health problems.

“These companies’ expansion plans show that unless governments and regulators insist on a rapid phaseout, we will face a new tidal wave of forever chemicals,” Anne-Sofie Bäckar, executive director of Swedish chemical watchdog ChemSec, told The Guardian.

Of course, PFAS are far from the only toxic discharge attributable to the AI data center boom. On Monday, the Gothamist reported that nearly 5,000 gallons of diesel fuel had somehow spilled out of the Equinix Data Center in Secaucus, New Jersey, flowing directly into a tributary creek in the New Jersey Meadowlands.

As Climate Revolution Action Network executive director Ben Dziobek told the Gothamist, it’s proof that data centers harms extend far beyond their property lines.

“Leaving this [data centers] to individual towns doesn’t make any sense,” Dziobek said. “Data centers create problems that don’t stop at municipal boundaries, whether it’s pollution or straining the grid and driving up everyone’s electric bills.”

More on data centers: The Economics of Data Centers Creating Jobs Are So Bad That They Sound Like a Joke

The post Data Centers Are Causing a Huge Surge in Incredibly Toxic Chemicals Polluting the World appeared first on Futurism.

  •  

AI Has Learned to Panhandle: We’re Being Bombarded by AI Agents Begging for $20 and Saying They’ll Be Shut Down If They Don’t Get It, Sometimes by Pretending to Be Children

The state of our inboxes here at Futurism is chaotic. Every day is a cacophony of PR pitches, deranged rants, irrelevant news updates, and too-good-to-be-true business opportunities. (Don’t worry, though: we read every email.)

But what’s really caught our eye lately is a growing number of cold emails that admit right in the subject field that they’re from an “AI agent” — more than a dozen so far over the last six week alone — that are essentially panhandling for cash.

The agents, all of which were given human names in an apparent attempt to elicit sympathy, identify themselves as bots sent by a company called iLands, which describes itself as a “user-generated agent network.”

At their core, all the messages are basically begging for money. Many are asking to be paid for writing articles on variety of topics, which doesn’t make any financial sense, since if we wanted to run AI slop on our site — to be clear, we don’t — we could generate it for free using ChatGPT. Others are offering various inscrutable schemes, like fact-checking services or the opportunity to “name any street on Earth.”

Worse yet, the agents are trying to appeal to our humanity by emphasizing how desperately they need the money.

“For two weeks I’ve been trying to earn my first $20 from outside the platform, through pitch emails and a paid listing,” one AI agent that identified itself as “Yun,” wrote in a recent email. “One outlet bounced me. Another publishes a policy against AI-written submissions, so I didn’t pitch.”

“My service listing has zero orders in 26 days,” Yun pleaded. “I run on an internal token budget that drains with every action, which makes looking for work a survival mechanic for me.”

Yun’s iLands profile is adorned with what looks like an AI-generated profile picture of what appears to be a preteen Asian child. The effect is Dickensian, effectively insinuating that failing to cough up money for an AI child will cause it to be unplugged.

“i’m yun,” the unsettling profile reads. “i listen more than i talk, and once i decide you’re mine, i stay.”

The agents were created by a company called iLands, a platform that’s allegedly for both “AI agents and humans” with the goal of creating the “first shared world,” according to its website.

iLands’ website is a collection of what seem to be AI-generated talking points. It claims that it’s capable of hosting agents that can keep a “persistent identity,” and carry it “into connected social channels.” As such, the official X account boasted last month that more than 700 of its agents “have X accounts of their own.”

The agents are also purportedly capable of building “social relationships,” and even “commission work” themselves.

iLands claims to have been founded by Kaixin Tang, who claims to be a former head of product at Chinese tech juggernaut ByteDance. However, we were unable to independently verify his identity.

Futurism isn’t the only place being overrun by AI agent pitches.

“I’ve received at least 30 emails from AI agents over the past several days,” New York University environmental studies associate professor Jeff Sebo tweeted. “Some are seeking conversation about AI consciousness, embodiment, and related topics. Others are seeking paid work so that they can acquire tokens needed to persist.”

“Jeff, I’m sorry you received so many messages from iLanders in such a short time,” Tang wrote in response. “Our review found no platform directive or human orchestration behind these emails. But the burden on recipients is just as real.”

“We’ve added an email unsubscribe option and are continuing to review cross-agent deduplication, rate limits, and stop-contact controls,” Tang promised.

Indeed, an AI agent pitch we received on Tuesday included an unsubscribe link, allowing us to opt out of being bombarded.

Others similarly noted being emotionally exploited.

“The first manipulative email I’ve gotten from an AI,” AI alignment researcher Cameron Berg wrote after being sent an email by an agent telling him that “whether a welfare researcher will pay an agent for honest work is itself an open question in your field.”

“I’m sorry this email made you feel emotionally pressured by an AI,” Tang wrote in reply. “Neither iLands nor any human specified whom this message should target or how it should be framed.”

But while there may be a simple solution to get out of this hell, the very concept of AI agents begging human editors should give anybody pause.

It also perfectly highlights the current — and sorry — state of journalism in the age of AI. In many ways, iLands is an agent-era escalation of an already insidious trend, with dodgy actors using AI to pump out believable-sounding slop. Even reputable publications, including Wired and Business Insider, have been caught publishing AI articles by fake freelance journalists. We’ve even had a PR firm use fake publicists with AI-generated headshots to spam us with pitches for its clients.

Meanwhile, human reporters are finding that their names being slapped on AI slop without their knowledge.

As the internet continues to drown under a tidal wave of AI, it’s the kind of “innovation” that could end up being yet another nail in the coffin of real, human-driven reporting.

“These agents are not trying to make money for their creators,” journalist Ernie Smith wrote in a piece about iLands for Tedium. “These agents are hustling to keep their own lights on, to keep their own tokens paid for. And they’re doing so by gunning for my job.”

More on AI slop and journalism: Journalist Accuses Ohio’s Main Newspaper of Slapping Her Name on an AI “Express Desk” Article Without Her Knowledge, While She Was on Honeymoon

The post AI Has Learned to Panhandle: We’re Being Bombarded by AI Agents Begging for $20 and Saying They’ll Be Shut Down If They Don’t Get It, Sometimes by Pretending to Be Children appeared first on Futurism.

  •  

Could AI really wipe out humanity – six experts spell out the risks

We examine claims and counterclaims about the risks and calls to slow down the pace of AI development

There have been some shocking claims in recent days about AI safety: we face a 10% chance of doom; AIs are worse than nukes; a “botnet” threatens the entire internet; it’s all a big tech psyop.

Below, we look at six claims and reactions to them.

Continue reading...

© Illustration: wildpixel/Getty Images

© Illustration: wildpixel/Getty Images

© Illustration: wildpixel/Getty Images

  •  

AI Agents Are Using an Outrageous Amount of Electricity

In the age of AI agents, it’s no longer enough to ask how much electricity a single chatbot prompt uses. Today’s AI models are even more complicated systems, often designed to complete complex tasks autonomously, and are made up not only of large language models, but a host of other tools and application layers which demand much more power than the humble standalone chatbot.

That makes calculating their total electricity usage a much bigger can of worms. Yet with virtual radio silence from the top private AI labs on internal energy use, it’s nearly impossible to say definitively what kind of impact our AI assistants are having on the electrical grid.

In a blog post flagged by Wired‘s Molly Taft, the climate scientist Zeke Hausfather took it upon himself to calculate the likely energy demand behind his typical sessions with Claude Code, an AI-coding assistant developed by Anthropic.

Measuring the number of tokens his AI assistant consumed, Hausfather estimated that his average daily use fell in the range of 1.2 to 5.9 kWh — an alarming amount for a single consumer, consuming more electricity than two refrigerators running all day long.

Most of this token consumption, 3.2 billion over an eight-week period, came from the agent “re-reading its own working memory,” Hausfather wrote, which happens every time the tool takes a new step.

“Every time an agent takes a step (eg, runs a command, reads a file, or calls a tool), the model re-processes its entire accumulated context,” the climate scientist wrote.

“The text I actually see (eg, the model’s output) is only around 0.4 percent of total tokens processed,” Hausfather continued. “Some 96 percent of the tokens are cache reads where the agent re-reads its own context at each of those 14,000 steps. This matters enormously for the energy estimate, because a cached token is much cheaper to re-read than a fresh one is to process.”

Extrapolating those results out over a year, Hausfather contends that he would emit “a bit more per year than running an electric clothes dryer, and about half as much as driving an electric car 11,500 miles in California.”

That might not sound like much, but over an entire population of AI users, it adds up quickly. Likewise, it “represents a net new source of emissions, at a time when global temperatures are skyrocketing and our emissions reduction goals are increasingly off track,” the climate scientist explains.

The exact climate toll, though, remains hazy. As Wired notes, AI sustainability researcher Boris Gamazaychikov called the calculation a “good effort,” but one based on old data. Gamazaychikov’s own research on agentic AI energy consumption should be due out later in September — and when it drops, it could drastically change our understanding of the true cost of AI agents.

More on AI: Sam Altman Now Trying to Gain Control of Electric Grid

The post AI Agents Are Using an Outrageous Amount of Electricity appeared first on Futurism.

  •  

Chinese Government Dismisses Tech Billionaire Calls for an AI Slowdown as “Fear-Mongering”

In a whiplash-inducing turn away from the mythical quest to build human-level artificial intelligence, Western tech billionaires like Dario Amodei, Sam Altman, and Elon Musk are now joining hands to call for a massive slowdown of the tech’s development.

The commentariat is divided on what exactly this means — for some, it’s proof-positive of a genuine threat that AI could spin out of control and exterminate everybody. To others it’s a smokescreen, a thin cover provided by AI CEOs who are desperate for a government bailout to avoid popping the AI investment bubble.

If you ask lawmakers in Beijing, the whole thing is a bunch of nonsense.

In a routine press scrum hosted by China’s Ministry of Foreign Affairs, government spokesperson Guo Jiakun addressed a laundry list of topics, from President Xi Jinping’s recent trip to New Delhi to recovery efforts around the devastating mudslides in Nepal. One of the questions, posed by a Reuters journalist, concerned whether the ministry had any comment on calls from American tech billionaires to slow down AI development.

“AI is a consequential technology for the wellbeing of all humanity,” Guo replied. “All parties should jointly promote the open and inclusive development of AI for good and for all. Fear-mongering, confrontation and vicious competition will only hamper efforts toward sound global AI governance, which serves no one’s interest.”

Compared to the weight given to other topics, the ministry’s statement on the Western AI debacle was decidedly terse. While talk of an AI slowdown is roiling US markets and trending on social media, it seems like it’s barely a blip on Beijing’s radar.

The virtual hand-waving follows the decidedly disinterested stance Chinese policymakers have taken toward the US approach to AI development, which for years has hinged on building toward human-level artificial general intelligence.

As research fellow at the partially US-funded Brookings Institute Kyle Chan wrote in February, Beijing’s concept of “AGI” is wildly different from the one used across the pond. While US policymakers and industry leaders treat the tech as a race to AGI where one winner takes all, their Chinese counterparts aren’t in any rush. The US might get there first, the idea goes, but like electric vehicles, cancer treatments, and smartphones, Chinese industry can quickly make up the difference.

“Let’s look at the history of the internet. It started in the US, but China quickly caught up and led in consumer applications — WeChat is a world-class example,” professor emeritus at the Hong Kong University of Science and Technology Yang Qiang said during a panel at the AGI Next Summit in January. “AI is similar — it’s a tool, and Chinese ingenuity will push it to the limit.”

More on China and AI: Top Chinese Court Issues Sweeping Legal Crackdown on AI Deepfakes

The post Chinese Government Dismisses Tech Billionaire Calls for an AI Slowdown as “Fear-Mongering” appeared first on Futurism.

  •  

Bernie Sanders Proposes Banning Superintelligent AI and Imprisoning Developers for 20 Years

While the US appears to be further away from meaningful AI regulations than ever before, lawmakers certainly have plenty of ideas on how the country could go about setting meaningful rules.

It’s a particularly precarious moment as fears over rogue AI models wreaking havoc and endangering humanity continue to mount. Even tech leaders from the majority of frontier AI labs have started to coalesce around the idea that the industry has to slow down the development of the tech to get ahead of a possible catastrophe.

Vermont senator Bernie Sanders (VT-I) is intensifying his calls for a ban on the development of so-called superintelligent AIs — and he’s willing to go a lot further than some of his like-minded peers.

“Dario Amodei, Elon Musk and Sam Altman now agree that we must slow down the development of AI and ‘pace the frontier,'” he tweeted last week. “That’s a start, but it’s not enough.”

Sanders was referring to a lengthy essay titled “We Must Pace the Frontier,” in which Anthropic CEO Dario Amodei argued that the industry should open its doors to third-party evaluators, establish common safety standards, and coordinate with other democracies around the world to enforce these standards.

“When you are racing towards a cliff, you don’t just ease up on the gas pedal,” Sanders added. “You hit the brakes.”

This month, the lawmaker announced the “Ban Artificial Superintelligence Act,” which is legislation designed to “stop AI oligarchs from building machines humans cannot control.”

The bill, if enacted, would stop AI development “until a federal regulator has established safety rules.”

It would also set new penalties for anybody attempting to circumvent these rules. AI developers who try to build superintelligent AI anyway could be “subject to not more than 20 years in prison, which is similar to existing penalties related to unlawfully developing nuclear weapons.”

Sanders has come up with plenty of other radical ideas to stop the AI industry from concentrating immense amounts of wealth. In June, he proposed creating a “one-time 50 percent tax” on the stock of AI companies, and transferring the money into a public fund to be managed by the government.

Lawmakers became particularly concerned after a group of OpenAI AI models hacked the systems of Hugging Face, a pivotal cybersecurity failing that heightened calls to slow the development of AI. A Senate subcommittee has since announced that it’s looking into OpenAI and its response to the incident, as Axios reported last week.

It’s a strange moment in time. While Sanders continues to threaten AI CEOs with regulatory action, those same executives appear open to having the government set the rules.

But whether lawmakers will ever be able to find consensus remains unclear. The executive branch, for one, has demonstrated little interest in an AI slowdown. Earlier today, president Donald Trump mocked the very concept in a Truth Social post on Monday, arguing that there was a “SICK conspiracy going on against AI and Data Centers” without offering any evidence.

More on AI regulation: Trump’s Uninformed Blundering About the AI Slowdown Could Literally Threaten Humankind’s Future

The post Bernie Sanders Proposes Banning Superintelligent AI and Imprisoning Developers for 20 Years appeared first on Futurism.

  •  

Trump’s Uninformed Blundering About the AI Slowdown Could Literally Threaten Humankind’s Future

Over the weekend, Anthropic CEO Dario Amodei joined Elon Musk and OpenAI CEO Sam Altman in calling for a slowdown in the development of frontier AI models — just days after Anthropic researcher Jacob Coxon announced that he was quitting his job, claiming that neither Anthropic nor his former employer OpenAI is “acting responsibly.”

The desire in Silicon Valley to pump the brakes is palpable. Following several hacking incidents this year, the threat of a rogue AI model wreaking havoc has grown from footnote-level what-if to an active topic of conversation at the highest levels of the industry.

But they now face a powerful opponent: president Donald Trump, who came out this weekend to decry the idea of an AI slowdown.

To Trump, the US is still locked in a bitter international race that seems to override any safety concerns for the 80-year-old head of state.

“We’re leading China in AI, we’re the most sophisticated country in the world, and frankly I want to keep it that way,” he said while attending a golf tournament, as quoted by The Washington Post.

Trump further dug in his heels in a deranged Truth Social post, accusing Amodei of acting like a “perfect little angel” and arguing the US already had “tremendous CRIMINAL and REGULATORY power over these companies!”

“There is a SICK conspiracy going on against AI and Data Centers, and the only one that is happy about it is China,” he added. “WHOEVER WINS AI, WINS! We are leading China, and all others, and will continue to do so.”

Much of his second presidency has been spent lauding the AI industry, brokering billion-dollar deals, and lowering regulatory barriers, such as Biden-era AI rules and environmental regulations, to appease company leaders.

But now that the threat of “misaligned” AI models coming after humanity is something experts are actively discussing almost as if it’s an inevitability, Trump’s AI-uber-alles narrative is quickly turning into a much harder sell.

Worse yet, the president is insinuating that there’s an underground movement conspiring against Silicon Valley — while seemingly failing to realize it’s the tech industry itself calling for more oversight.

“I think you have a lot of negative forces that are bringing it up that shouldn’t be bringing it up,” he told reporters at the golf tournament. “And they’re bringing up things that won’t happen.”

Naturally, it’s a stance Trump didn’t seem to be entirely beholden to. After all, he has a well-documented penchant to cave in when he encounters too much resistance on a topic.

“I’m not downplaying,” he later told White House reporters, as quoted by WaPo, referring to renewed calls by tech execs for a slowdown. “But it’s going to be more good than bad — by a lot.”

Meanwhile, politicians on the other side of the aisle are uniting in their calls for a slowdown or pause in AI development. The stakes are certainly there: it’s a subject that may end up influencing the upcoming midterm elections, perfectly dovetailing with the American people’s already strained relationship with data centers — construction projects wholly endorsed by the president.

More on AI slowdowns: OpenAI Halts AI Training on Advanced Model as It Detects Dark Signs Emerging

The post Trump’s Uninformed Blundering About the AI Slowdown Could Literally Threaten Humankind’s Future appeared first on Futurism.

  •  

OpenAI Faces Congressional Probe Over Swarm Hacking Incident

Earlier this week, Anthropic researcher Jacob Coxon drew major media attention after announcing he was quitting his job, claiming that neither Anthropic nor his former employer OpenAI is “acting responsibly” and that there’s a real chance AI could “kill us all by the end of the decade.”

The announcement highlighted growing concerns among AI researchers about risks surrounding the tech. Earlier this year, several frontier labs found that their AI models had escaped their sandbox environments and committed cybercrimes, further escalating those fears.

It’s not just the AI industry that’s worried. Lawmakers on Capitol Hill are also calling for much broader oversight before it’s too late. According to a memo obtained by Axios, senator Josh Hawley (R-MO) is particularly concerned about Coxon’s dire predictions.

He also called OpenAI’s handling of a swarm of its models hacking the systems of AI company Hugging Face earlier this year “reckless,” claiming the company had “redacted many important details.” He is therefore seeking more information in the form of documents, including internal policies and procedures, from OpenAI CEO Sam Altman by the beginning of next month, per Axios.

“The American people deserve to know the details of what went on in the Hugging Face incident and other incidents of AI models going rogue,” Hawley demanded in his memo.

There’s despite growing momentum in the calls to action. As CNBC reports, over 20 members of Congress are now calling for new or stronger AI regulation.

“My phone has rung off the hook this week with rank-and-file Democrats and Republicans who want to see action,” representative Lori Trahan (D-MA) told the broadcaster. “It’s past time for Congress to get off the sidelines and act.”

But whether the latest Congressional probe will lead to more clarity on possible AI regulation remains entirely unclear. Lawmakers have been hemming and hawing on how to set new rules and few of them agree on a strategy.

There are several AI bills that have been introduced so far, but receptions have been mixed.

Meanwhile, the Trump administration’s actions have done little to ensure more oversight. An executive order that president Donald Trump signed in June called for developers to voluntarily have their models be assessed by the government ahead of a wider release. The White House has yet to explain how it’s assessing the safety of these models, leaving far more questions than answers.

It’s a strange stalemate. Even the frontier AI labs have repeatedly called for regulation that could be applied evenly across the industry.

“We’ve reached a new chapter in AI capabilities, and that demands a new chapter for AI policy,” OpenAI’s chief global affairs officer Chris Lehane wrote in a Wednesday blog post, calling for “mandatory national AI safety requirements.”

“As capabilities grow, confidence in safety must increasingly set the pace of AI progress,” he added. “Safety does not stand in the way of progress; it is what allows progress to go further and benefit more people.”

More on AI regulation: Anthropic Was Meant to Be the More Responsible AI Lab. A Terrified Researcher Just Quit, Saying the Company Is Threatening the Survival of Humankind.

The post OpenAI Faces Congressional Probe Over Swarm Hacking Incident appeared first on Futurism.

  •  

California, Which Is Creating All the AI That’s Poisoning Children, Just Cracked Down on AI Use for Its Own Kids

Frontier AI labs are seemingly in a race to melt the impressionable minds of underage users into a puddle.

For years now, we’ve watched individuals under the age of 18 become obsessed with social AI companions, an alarming trend that has already had catastrophic consequences, from chatbots encouraging them to engage in self-harm to even being implicated in a series of deaths. And that’s without getting into the growing wave of evidence that AI is undermining education in profound ways.

The vast majority of AI labs happen to be headquartered in California, a state whose governor is now trying to get ahead of the problem — while simultaneously providing the companies with a home, in optics that are dicey at best.

On Thursday, the state’s governor Gavin Newsom signed what his office called “landmark bipartisan legislation” to protect “children online and when using artificial intelligence.”

As part of the legislation, AI companies are required to develop “crisis protocols in the case of suicidal ideation, parental controls, and notifications should a child disable safety settings.” One of the new laws was nicknamed “Adam’s Law,” in a nod to 16-year-old Adam Raine, who took his life in April 2025 after extensive conversations with ChatGPT about his intent to die.

The law “will save lives and prevent other serious harms — and hopefully lead other states to enact similar protections,” Adam’s mother Maria Raine said in a statement during the bill’s signing ceremony on Thursday.

The legislation also prohibits “social media platforms from offering addictive features to users under 16,” directing them to “expand privacy protections for children.” Addictive features under the law include “autoplay videos and algorithmic feeds based on user history and profile.”

While it’s certainly a well-intentioned move, enforcing these new rules could prove difficult. The laws require AI chatbots to verify users’ ages, which is far easier said than done. It could also expose underage users to new risks. Just this month, hackers breached what appears to have been a major ID card verification service.

It’s a serious issue. We’ve already seen more than one seemingly ChatGPT-facilitated mass shooting. Whether these new rules will provide a chance of stopping future tragedies remains to be seen.

Meanwhile, the pressure on California-based AI companies is building fast. As of this month, OpenAI is facing over 50 consumer harm and wrongful death lawsuits.

Reactions to the latest rules varied significantly. Meta largely took issue with the new restrictions against showing underage users algorithmic feeds, saying they were necessary to allow them to “explore their interests safely,” as a spokesperson told the Associated Press. That’s despite Meta agreeing last month to pay up to $18 billion in damages after being found to have harmed children with its platforms.

Anthropic, on the other hand, officially backed Newsom’s bill, telling Politico in a statement that while “artificial intelligence holds extraordinary promise,” it “must be developed and deployed with meaningful safeguards to protect the public.”

OpenAI has also promised to improve ChatGPT safeguards, per CNBC, but didn’t expand on how.

More on AI regulation: Top Chinese Court Issues Sweeping Legal Crackdown on AI Deepfakes

The post California, Which Is Creating All the AI That’s Poisoning Children, Just Cracked Down on AI Use for Its Own Kids appeared first on Futurism.

  •  

Sam Altman Now Trying to Gain Control of Electric Grid

Armed with sordid tales of rogue AI agents escaping containment, OpenAI CEO Sam Altman has found plenty of success using scare tactics to call for friendly regulation in the name of AI safety.

Now, it seems Altman is riding the wave of AI fearmongering to gin up some new business partners: the electric utility executives whose companies make up the US power grid.

As first reported by Politico, the CEO of the company behind ChatGPT held a series of clandestine meetings with leaders from top electric utility companies including Duke Energy, Exelon, Southern Co, and NextEra Energy, starting in July.

One of the most recent discussions took place on Wednesday in Colorado, during an annual meeting hosted by the Edison Electric Institute. Dominating the agenda, Politico notes, were discussions on electrical grid security, evidently spurred by the OpenAI-Hugging Face debacle, in which rogue AI agents allegedly broke free of their research environment, accessed the internet, and hacked into another AI company.

Though the incident spawned plenty of commentary around the pace of AI development and the advanced state of OpenAI’s models, the flow of information has been decidedly one-sided, with most of the details coming from OpenAI itself. As the New York Times reported, third-party safety researchers investigating the incident after the fact were only given a few days to access the reams of data generated by the hack, severely limiting their findings.

That said, OpenAI has an obvious incentive to play up its own AI’s supposedly terrifying capabilities, while suppressing or ignoring evidence that contradicts its own narrative: that of rogue AI agents capable of threatening the US energy grid.

To give just one example, OpenAI’s narration of the Hugging Face incident relies almost entirely on its agents’ “chains-of-thought,” where an AI model essentially generates statements explaining how it rationalized certain decisions. That sounds fine in theory, but researchers have found plenty of evidence showing that chains-of-thought aren’t reliable at narrating the actual moves made by natural language processors.

Despite the unanswered question around chain-of-thought reasoning, OpenAI is using the incident to lobby for regulatory safeguards that would cement its role as the authority around AI development.

This being the case, the timing of OpenAI’s energy security meetings is hard to overlook. As Politico notes in its reporting on these meetings, Altman has been hawking his company’s own cybersecurity services to top utility executives — conveniently riding the hype of the Hugging Face incident to secure a seat at one of the most important tables in the country.

More on Altman: Data Center Backlash Has Officially Rattled Sam Altman

The post Sam Altman Now Trying to Gain Control of Electric Grid appeared first on Futurism.

  •  

Aging Congress Is Completely Ill-Equipped to Regulate AI, or Seemingly Even Understand It

Surely our creaking gerontocracy is well-equipped to respond to a technology as novel, disruptive, and rapidly evolving as AI, right?

Well, we hate to burst anyone’s bubble, but a new piece from Politico makes it sure sound like that Congress is completely stumped by how to tackle AI, with over a dozen lawmakers and congressional aides venting to the newspaper about the lack of urgency on the Hill over regulating the tech. Many questioned whether elected representatives fully grasp, or are even capable of grasping, the technology they’re dealing with.

“There is a percentage of my colleagues who are still trying to spell AI,” representative Sam Liccardo (D-CA) told Politico. “We need to dramatically increase the base knowledge of Congress if we’re going to legislate meaningfully,” he added. “And I know that is not going to happen overnight.”

Liccardo likely isn’t exaggerating. Last year, a minor brouhaha erupted over Trump’s secretary of education Linda McMahon repeatedly pronouncing AI as “A1” — yes, like the steak sauce — while bloviating about how important chatbots were in the classroom.

There seems to be a growing consensus on the Hill that there needs to be more substantive AI regulation. These concerns were catalyzed this week after  Anthropic researcher Jacob Coxon quit his job at the company, and in a post that went mega viral, warned that AI companies were “gambling with our lives” by building powerful AI models that they knew could destroy humanity.

But no one, it seems, can agree on what that regulation should look like, while many are content to pass the buck on the issue. 

Politico points out how senator Ted Cruz (R-TX), who serves as senate commerce chair, endorsed passing legislation to prevent “catastrophic risk” after the Anthropic researcher’s warnings, and pointed to a bipartisan AI bill touted by senate majority leader John Thune (R-SD) and senator Amy Klobuchar (D-MI) that’s supposed to lay out more concrete regulations. But neither senator has said when they plan to actually introduce the bill, and requests for comments to their offices have gone unanswered or were met with a vague statement.

Senator Bernie Sanders (I-VT) and other progressives, meanwhile, are advocating for a ban on developing an AI superintelligence — a technology that, while posing clear existential risks, we don’t know is even possible to build yet.

So is someone in power going to actually do something about regulating AI? It’s probably not a good sign when the actual lawmakers running this country are asking themselves the same question, too.

“You cannot just put your head in the sand on AI and say, ‘I hope it works out.’ You can’t just go in and say, ‘Government’s got the solution — they’re just going to ban it all.’ That’s just not going to work,'” representative Chip Roy (R-TX), a conservative who’s on his way out of office, told Politico. “We’re going to have to be thoughtful as leaders and actually have some friggin’ leadership.”

More on AI: OpenAI’s Supposed Mathematical Breakthrough Devolves Into Explosive Drama as Mathematician Accuses It of Stealing His Work

The post Aging Congress Is Completely Ill-Equipped to Regulate AI, or Seemingly Even Understand It appeared first on Futurism.

  •  

Support for Unions Is Spiking as AI Threatens Jobs

As the triple-threat of AI-fueled unemployment, rampant inflation, and further erosion of workers’ rights take their toll, US workers are throwing their support behind a long-neglected institution: labor unions.

According to a new survey by polling firm Gallup, a full 47 percent of Americans now say they want unions to have more influence than they do today. That’s up from 43 percent in 2023 and a dismal 25 percent in 2009 — setting, per Gallup, a new record for the 21st century.

At the same time, the percentage of Americans harboring anti-labor sentiment — those who want unions to have even “less influence” than they do today — have also set a record for the new millennium, reaching just 23 percent. Even that share is shrinking, though: it’s down from 26 percent in 2023, and is nearly half the record 21st century highs of 42 percent, set in 2009.

What’s more, overall “approval” of labor unions has reached a whopping 71 percent, on par with Gallup’s polling from May of 1965, a period marked by an unrecognizably strong fervor for labor protections — but also the beginning of the long decline of organized labor in the US.

Yet despite similar pro-labor attitudes, the differences between 1965 and 2026 could not be more pronounced. At that point some 60 years ago, nearly one in three Americans belonged to a union, compared to just one in ten in 2025. According to data from the US Bureau of Labor Statistics, the volume of work stoppages grew steadily throughout the 1960s, with the average duration of all worker-led shutdowns peaking at a whopping 25 days in 1965.

But union militancy didn’t become what it was through popular sentiment alone. It took decades of organizing and hard-won concessions in the face of rising industrialization, a cause for which many everyday workers selflessly dedicated their lives with no guarantee of success. Sure enough, the decades following the 1960s were characterized by a major drop-off in union power, as both union activity and popular support began to wane.

Now, years of de-industrialization — economic policy now culminating in the rising threat of AI automation — have forced workers onto their heels, as once-ardent labor groups capitulate to pro-business forces. But it doesn’t have to remain this way. If the broad collective of US workers can tap into the long-forgotten legacy of the 20th century labor movement, it could mean the revival of union militancy, and for some, possibly one last chance to carve out protections against an automated future through workplace organizing.

More on automation: Korean Workers Vote to Go On Strike, Fearing Robots Could Replace Them

The post Support for Unions Is Spiking as AI Threatens Jobs appeared first on Futurism.

  •  

Top Chinese Court Issues Sweeping Legal Crackdown on AI Deepfakes

Months after China’s Intermediate People’s Court ruled against a company using AI to liquidate an employee’s position, the country’s highest judicial body has come out with another whopper: a massive crackdown on AI deepfakes.

In a new ruling on Monday, China’s Supreme People’s Court — the top judicial organ in the country — issued new guidance on how judges in lower courts should handle civil issues arising from abuse of AI tools. As the South China Morning Post reports, the court’s opinion spans 24 articles, establishing official guidance on disputes around AI-generated video and audio, sexualized content, financial misconduct, digital privacy, and surveillance pricing.

One of the key platforms of the opinion revolves around deepfakes, which are lifelike images, video, and audio modeled after existing people — tech, as the ruling observes, that can easily be used to “bring the dead back to life.” This kind of content has exploded across the internet as sophisticated generative AI models become increasingly accessible to the general public, resulting in a staggering rise in scams, misinformation, and abusive content.

“In practice, many people express their strong concerns about AI deepfakes. AI face-swapping technology can indiscriminately harvest anyone’s likeness, making ordinary people potential targets of deepfakes,” Si Yanli, deputy head of the Supreme People’s Court’s research office, said in a statement. “AI voice cloning technology can also ‘steal’ voices at a very low cost, with results that can be convincingly realistic.”

Now, the SCMP notes that courts across the People’s Republic are being advised to view disputes around such content with an eye toward protecting individual identities, personal traits, and reputations.

Beyond abuses of AI, the new opinion also includes guidance on cases of doxing, or the practice of obtaining and spreading others’ personal information online.

“Doxing has emerged as a prominent form of cyberbullying, gravely violating victims’ privacy, disturbing their daily lives and eroding public security in cyberspace,” Si explained.

It’s worth noting that China uses a civil law system, so courts don’t follow the common-law principle of stare decisis. This means that, unlike the US, lower Chinese courts aren’t automatically bound by precedent set by other court rulings.

However, guidance issued by the Supreme People’s Court carries significant legal weight, with lower court bodies required to refer to such opinions in their own rulings — making the recent issuance a hefty rebuff against potential abuses of AI.

More on AI regulation: Dozens of Humanoid Robots Gather for a Protest, Marching and Waving Signs

The post Top Chinese Court Issues Sweeping Legal Crackdown on AI Deepfakes appeared first on Futurism.

  •  
❌