The notion that scammers can use AI to sharpen their deceptions, polish their language, and lubricate their banter with victims is now a reality for anyone fighting the fraud operations that steal tens of billions of dollars a year worldwide. But can AI fully replace a human scammer, autonomously building the web of deception leading up to the fake investment that defrauds the mark? One study's experiment suggests that it can—and may even be able to carry out the majority of that long con more effectively than humans.

Researchers from four universities—Amrita Vishwa Vidyapeetham in India, Foscari University of Venice, the University of Melbourne, and Ben Gurion University of the Negev—carried out a broad study on the use and potential of generative AI chatbots in the growing scam industry centered around a form of fraud known as “pig butchering,” text-based romance scams that eventually shift to fake crypto investments that steal as much as six-figure sums from victims. In their study, the researchers pitted AI chatbots directly against humans in a simulation of the scamming process—or more specifically, the long, trust-building conversations that eventually lead up to soliciting a fake investment from the scam’s target.

They found that for the relationship-establishing stages of the scam—the stage that in real-world scams typically represents the longest part of the interactions with the victim, often stretching to months—an AI chatbot performed remarkably effectively, successfully impersonating a human and by some measures outperforming the real human “scammers” in their experiment.

Read full article

Comments

Marcus Mendes:

Last April, Brazilian officials notified Apple and Google over an alleged lack of age controls in betting apps on their respective app stores.

9to5Mac:

Brazilian users browsing the App Store rankings in categories such as Navigation, Travel, and Weather have noticed a growing number of poorly made games appearing among the top results, many of them featuring AI-generated illustrations of animals as their app icons. Many carry an AL age rating, making them available to users of all ages, including children.

As it turns out, these are so-called jacket apps, which are just a front for hidden betting and gambling apps.

[…]

The repository also includes explicit instructions intended to prevent the apps from being flagged as suspicious during App Store review. These include giving each app uniquely named startup and remote-configuration codes, making it harder for reviewers to identify them as part of the same group.

Ironically, the App Store’s own recommendation system appears to have little trouble grouping the apps together, as the “You Might Also Like” section on several of these apps points users toward other suspicious apps more frequently than they point to genuine apps.

John Gruber (Mastodon):

Obviously the main problem is that these apps disguise themselves Trojan Horse-style, and slip through App Store review undetected. I don’t think it’s reasonable to think App Store review could catch every attempt at this sort of thing though — if the horses are disguised well-enough, some of them are going to slip through. That’s actually fine.

I agree that it’s unreasonable to expect App Review to catch all those, but then Apple should not be pretending that it can. This was all obvious from day one. The public perception of App Review is completely at odds with what it actually can and does do.

But the first example 9to5Mac cites isn’t well disguised at all. What sense does it make that “PotatoPlot Puzzle” — a game featuring a rabbit potato farmer — is a weather app? How is that not a red flag right there?

[…]

I always mention that the point of such a squad shouldn’t be to make the App Store 100 percent scammer-free — which I believe is an impossible goal that would set the team up for failure. The point should be only to make the App Store free of successful scams. Focus on the most popular apps and the most lucrative apps.

The App Store is more than 18 years old. The idea of patrolling the top charts probably occurred to Apple before it even shipped. They just don’t want to do it.

Nick Lockwood:

Apple only took 24 hours to approve my iOS app update that crashed instantly on launch. Such a blessing to have that extra level of reliability and security that only a manual review process can provide 🙏

Incidentally, the crash was due to a missing dylib - something that could easily have been detected by the automated scan process, if only that scan were designed to protect users’ and developers’ interests instead of Apple’s.

Kyle Hughes:

There is simply no other way to maintain a modern, healthy software ecosystem without making sure that apps that have been around for 7 years and 20 major versions can’t get a build looked at within 48 hours. There’s no other way. Probably needs to be longer, honestly.

I had to file an Expedited Review Request because this update is a legitimate emergency. Four hours later, App Review rejected the build because they didn’t follow the first sentence of the testing instructions (which haven’t changed in 7 years). Back to Waiting for Review! There’s no other way.

Previously:

• Chrome Browser Default at the Top of the Mac App Store
• Sparrow Wallet Lawsuit
• Sirius Pomodoro
• The App Store Scammer Strikes Back
• Brazil: App Marketplaces and External Payments
• Scam Authenticator App Steals QR Codes
• Sketchy ChatGPT App Soars Up App Store Charts
• Gambling Ads on App Store Product Pages
• Schneier on Sideloading
• The App Store Isn’t Catching the Most Egregious Scams
• Scammy KeyWatch and Trezor Apps
• Widgetsmith and The Case of the Missing App Store Bunco Squad
• Big Fish Casino
• Apps Removed in Ban of Gambling-Related Apps

Update (2026-07-30): Kyle Hughes:

We got rejected in the Expedited Review again, this time for not having an account deletion feature. Except we do, and it’s one button press away from the screen that the reviewer was on. And it has always existed. Is this how the expedited review works? Malicious incompetence?

Update (2026-07-31): Kyle Hughes:

If you were wondering how long one of the “you can ask us to approve this build and can fix this issue in the next release” App Review errors takes to resolve when you’re in the Expedited Review Queue… me too! So far, 18 hours. (Ignoring the fact that we have the feature the rejection was for.)

Kyle Hughes:

Certainly it means “within a week,” right? “Within a month,” please? “In your lifetime,” I can only hope.

Joe Rossignol (PDF):

The complaint alleges that Apple violated California’s Consumers Legal Remedies Act and other laws by failing to adequately review and monitor apps distributed through the App Store, despite marketing it as “a safe and trusted place to discover and download apps.” Due to these assurances in Apple’s marketing, the complaint said the plaintiffs trusted that a Sparrow Wallet app they downloaded from the App Store on iOS was legitimate, when in reality it was a fraudulent spoof designed to steal Bitcoin from users.

Sparrow Wallet is only available on Windows, macOS, and Linux. The popular Bitcoin wallet is not available on iOS, but scammers have released fake lookalike apps in the App Store that can trick users into giving up control of their funds.

[…]

Even when victims reported fraud, the lawsuit alleges that Apple often took little to no action, and it says other fake Sparrow apps remain in the App Store.

Sparrow Wallet developer Craig Raw has publicly addressed Apple’s slow response to multiple fake versions of his app appearing in the App Store. In a January 2024 social media post, for example, he said “there is still a scam ‘Sparrow Wallet’ app on the @Apple App Store, despite myself and others having reported it weeks ago.”

Craig Raw:

My attempt to protect users from scam apps on the @AppStore has gotten my Apple Developer account flagged for termination - ironically, for “dishonest activity”.

[…]

I’m the developer of the real Sparrow Wallet, a desktop app, and I hold the registered US trademarks for the name and logo. I have publicly warned @Apple and the community about these fake apps from early 2024, but they keep appearing.

The app @Apple flagged was a placeholder that was never published. Its only purpose was to warn users that Sparrow is desktop-only and that other “Sparrow” apps aren’t mine. This approach may have been misguided, but there was nothing dishonest about it.

Joe Rossignol:

In response to our request for comment, Apple said it has taken swift action to remove any apps impersonating Sparrow Wallet[…] Apple says it takes immediate action against apps that do not comply with its guidelines.

Previously:

• Hide My Email Vulnerability Exposes Real Address
• Sirius Pomodoro
• The App Store Scammer Strikes Back
• Sanctioned Entities Slip Through App Store Controls
• Some of the Apps in the App Store
• Fake Bitcoin Wallet in the App Store
• Fraudulent Rabby Wallet App

Update (2026-07-28): Sarah Perez:

The complaint targets one of Apple’s longtime competitive arguments: that its tight control over its App Store makes its platform safer than those offered by its rivals. The company has used this claim to push back against deregulation of the app ecosystem, including third-party app stores and sideloading.

“As part of a sustained marketing campaign, Apple has positioned itself, its products and services, as offering a level of security and trustworthiness superior to any competing technology company,” the new filing states. “This includes assurances about the safety of applications distributed through its App Store. By retaining exclusive control over which apps are permitted on Apple devices, Apple has structured its platform to ensure that consumers depend entirely on its promise of safety and reliability,” it reads.

With third-party app stores, there would be an incentive to do better vetting than Apple, and with sideloading it would be clear that the app was not being downloaded from sparrowwallet.com.

Update (2026-07-29): Jeff Johnson:

What struck me about Apple’s response is that it places the burden of curating the App Store on developers and users rather than taking responsibility for it. After all, Apple reviews every version of every app before it can be published to the App Store. If Apple is curating the App Store, then why do scams have to be reported and removed? One wonders how the scams get past Apple review in the first place, and even if Apple review “is not perfect” (a gross euphemism), how the second line of defense, Apple employees patrolling the App Store for scams that the review team missed, also fails. The burden of App Store curation should not fall to me, a lone indie developer.

Apple’s response also ignores that the indie developer did report the scams, multiple times.

Previously:

• Hidden Gambling Apps in Brazilian App Store