Apple’s executives have been taking questions, hosting seminars, seemingly working around the clock to stress one very important thing: Apple is not using a white label version of Google Gemini to make Siri AI happen. They just pooled resources to get there. 

The new Siri AI is faster, more accurate, offers powerful contextual capabilities and shows how Apple has leap-frogged into a good peer position in an AI race critics felt it had already lost. Its market scale — even without the EU — is huge. For most consumers, Apple Intelligence and Siri will continue to be their primary/first engagement with artificial intelligence on a device.

Getting there took a lot of work, and Apple needed Google to get it done. Though there is still some confusion about what that means, Apple’s software chief tried to explain it this week. “We use none of the models that Google deploys to their customers, nor do we use the infrastructure and means by which they employ models to their customers,” Craig Federighi said in a presentation at WWDC.

Apple is not even using Google Search as the foundation of its system, he said. “This is the amount of Google Assistant we use,” Federighi said, pointing at an empty chart. “Nothing.”

Apple not Gemini, Siri AI is not Google’s

What makes this hard to understand is that we all know Apple partnered with Google to build Siri AI; back in January, we were told the next generation of Apple Foundation Models would be based on Gemini models and cloud technology. So, how can we have moved from partnership hero to usage zero?

The answer is, we didn’t. What happened is that Apple built its new Apple Frontier Models (AFMs) (the AI inside Siri AI) by training them using proprietary Apple data and reinforcement learning and then refined those models using “outputs from Google’s Gemini Frontier models.”

In other words, Apple used Google Gemini to help improve its own models, which means the models themselves, the AI in Siri AI, are Apple’s — but they were trained with help from Gemini. They are not white label iterations of those Google models. 

Apple also hit a second snag. Its very best model (AFM 3 Cloud Pro) requires more processor power to run than Apple could deliver using its own cloud-hosted Private Cloud Compute servers. Now, we know Apple doesn’t like using other people’s stuff. But it’s a realistic company that understands it sometimes must, and just as it uses AWS to support some of its services, it moved to adopt Google cloud services and Nvidia processors to drive the most demanding requests.

A question of trust

Apple also developed a technological solution that means it can claim the interaction remains just as private as if it were run on your device. Apple has made it possible for independent security experts to confirm this and says it is the only company that can deploy software on those servers, with strong security to ensure your device only interacts with those servers when you want it to. So far, no one has broken this protection. 

I came across an interesting report in which Tekonyx Founder and Chief Research Officer Sid Nag explained the significance of Apple finding a way to expand its Private cloud Compute infrastructure beyond its own data centers.

“Apple is effectively arguing that trust in AI systems should come from cryptographic and architectural guarantees rather than trust in the cloud provider itself,” he told Fierce. Where enterprises have faced a choice between access to powerful AI or privacy, Apple has introduced a new solution he called “portable trust.” This could conceivably become a new IaaS offering from the companies involved over time.

Working together for the benefit of all

So, while Apple’s models were built with Google’s help, and while its most advanced models run with support from Google and Nvidia, the models are Apple’s alone. 

It’s good for Google, of course. Apple is paying for this use, which helps the search giant claw back some of the value of its massive, muti-billion-dollar AI infrastructure investment. It’s not clear how much Apple is paying; earlier this year, the $1 billion figure was bandied around. But Apple’s decision to tie usage to iCloud subscriptions in some hitherto undisclosed way hints that the deal may also see some token-based usage charges on top of the basic Apple fee. I’ve not come across any details, but that’s what I surmise based on the size of Apple’s ecosystem and the growing realization of how quickly users can consume AI capacity.

What AI models is Apple running?

AFM 3 Cloud Pro is one of five Apple Frontier Models driving Siri AI.  Here’s how Apple describes those five models:

On-device models

• AFM 3 Core, the next generation of Apple’s 3-billion-parameter dense model. You’ll use this for basic text generation, summaries, conversational replies, all the standard uses. It can also handle indexing, search, App Intents, basic dictation and contextual awareness.
• AFM 3 Core Advanced, the most powerful on-device model. This is what makes Siri’s voice match mood or context, does all the high-accuracy dictation, and can process various data to handle tasks across different apps. It’s the AI driving your more involved Siri conversations. 

AFM Core Advanced is impressive in its own right, because Apple has managed to cram a 20-billion-parameter model onto a smartphone. It has done this by using a sparse architecture, which means it activates just 1 to 4 billion parameters at a time depending on the request. It is, however, only available to Apple’s most powerful systems — iPhone 17 Pro, iPhone 17 Pro Max, iPhone Air, M4 or later iPad, M3 or later Mac with 12GB+ memory.

Server-based models

• AFM 3 Cloud, which Apple calls its server-side workhorse, optimized for speed, efficiency, and performance.
• ADM 3 Cloud (Image), for image generation and editing, which unlocks advanced photo-editing tools, the all-new Image Playground, and more.
• AFM 3 Cloud Pro, the most capable server-based model, which powers the most demanding use cases, like agentic tool use and complex reasoning.

All five were built using the same common foundation, which was then specialized to reflect the proposed use of that model. It’s interesting to look at the human evaluation tests Apple ran to test how well these models performed; they demonstrate impressive improvement on the company’s original models, effectively justifying the decision to work with Gemini.

You can follow me on social media! Join me on BlueSky,  LinkedIn, Mastodon and subscribe to The Core.

Several trends are now converging that threaten to pit tech companies against the general public. 

Miniaturization has finally enabled companies to build AI glasses that look and function like normal glasses, but with microphones and cameras. People are increasingly talking to AI, rather than typing. And multimodal input, especially video, is on the rise. 

Put all of these trends together and you get a nascent industry pushing toward all-day, everyday AI glasses with cameras — and a worried public already pushing back at  the idea.

Let’s look at how we got here.

Meta started it with a surprise hit: its second-generation Ray-Ban Meta glasses, which later gained multimodal AI capability. Its Meta Ray-Ban Display glasses add one in-lens screen — but both versions of the glasses have cameras. (The company is working on a third generation that will probably ship next year.)

Google provides the AI and software platform through Android XR and Gemini, partnering with hardware makers to put its AI on other companies’ glasses. At Google I/O last month, Google unveiled frames from Gentle Monster and Warby Parker running Android XR with Gemini AI; they’re scheduled to launch this fall. Google is working on two types of AI glasses, one with screens and one that is audio-focused. Both types have cameras, though. 

Samsung is working to launch AI-powered smart glasses, too, code-named “Jinju.” The company offered up details at Google I/O alongside Google. The glasses feature a 12-megapixel camera with autofocus; run on Android XR with Gemini AI; are co-designed with Gentle Monster and Warby Parker; and are slated to launch in July at the Samsung Unpacked event. 

(As with Meta and Google, Samsung is working on AI glasses with and without screens, but both of its models have cameras.)

Tech giant Apple is also on the glasses train, based on reporting from anonymous insiders. Codenamed N50, the Apple glasses could have two cameras, one for pictures and videos, the other for multimodal AI input and hand-gesture control. (Apple is also working on a pendant and next-gen AirPods, both of which have cameras.) 

There’s Amazon, which is reportedly developing a new line of consumer AI glasses with a camera after its earlier, camera-less Echo Frames and Carrera Smart Glasses lines failed. (My guess is the problem was Alexa, not the lack of cameras.) Although its Echo Frames have been effectively discontinued — displayed as sold out online — the company is already testing AI glasses with cameras for enterprise use on hundreds of US-based Amazon drivers. 

Huawei in April launched its AI Glasses for the Chinese market — the lightweight glasses sport a dual-engine AI architecture and integration with its HarmonyOS ecosystem. It’s joined there by Xiaomi’s AI Smart Glasses, which are powered by the company’s HyperOS ecosystem and have cameras for photos and videos and for for reading QR codes. 

Beyond those well-known firms, other companies are making daily-wear AI glasses with cameras in them, including XREAL, Rokid, TCL, Solos, and Brilliant Labs.  

A minority of other companies is focused on glasses without cameras, including Even Realities (G1 and G2); MIRA (MIRA glasses); Dymesty (Dymesty AI glasses); Lucyd (Lucyd Lyte); and Huawei (Eyewear 2).

Get the picture?

Clearly, by the end of the year, the market will be flooded with all manner of AI glasses designed for everywhere, everyday wear. They can use prescription lenses or serve as sunglasses — and most of them will have cameras built in for photos, videos and multimodal AI. 

There’s just one problem: The public hates AI glasses with cameras.

Return of the ‘Glassholes’?

As we learned from Google Glass, a lot of people feel uncomfortable with a camera pointed at them while they’re talking to someone. And that backlash is back with the current generation of AI glasses. 

Because Meta is the market leader in the US, its Ray-Ban Meta glasses have borne the brunt of early disaffection. 

Texas Attorney General Ken Paxton recently launched a formal investigation into Meta’s AI glasses, calling them “a privacy nightmare for Texans,” claiming the devices “can easily invade personal privacy by collecting biometric data and recording Texans without their knowledge or consent.” 

Paxton also claimed the LED light on the glasses, which is designed to alert others that the camera is taking pictures or videos, can be easily defeated. In fact, some modders-for-hire charge up to $100 to physically destroy the LED and TikTok videos describe how to disable or cover the light. 

The pushback is happening elsewhere. Philadelphia courts banned smart Meta AI glasses with recording features from city courthouses and a petition is circulating to ban them from New York City bars and restaurants. MSC Cruise Line banned smart glasses in all public areas. And restaurants, gyms, and workplaces have begun banning smart glasses because of the camera. 

Uncertainty drives some of the concern. People don’t know whether they’re being recorded, and if they are, they don’t know who will see the video. It turns out, those  suspicions might be warranted. 

In February, Swedish publications Svenska Dagbladet and Göteborgs-Posten published an investigation that found Meta contractors in Kenya were reviewing footage from Ray-Ban Meta smart glasses — including “bank details, sex and naked people who seem unaware they are being recorded.” 

The New York Times published an internal Meta memo in February describing plans to add facial recognition (“Name Tag”) to Ray-Ban Meta glasses. The memo said the “political tumult in the United States would distract critics from the feature’s release.”

Then earlier this month, WIRED discovered dormant facial-recognition code called “NameTag” hidden inside Meta’s AI companion app. The code would let Ray-Ban Meta glasses identify strangers by face, a feature Meta publicly claimed “does not exist.” Meta quietly erased the code with an update one day after the exposé was published.

A coalition of civil society organizations wrote Congress to demand that Meta abandon its Name Tag facial recognition plans, calling it a “creepy and unacceptable escalation of surveillance.” The letter warned the technology could be adopted by law enforcement to surveil immigrants, people of color, and nonviolent protesters.

Finally, a range of reports involving AI glasses with cameras in them has emerged in recent months involving secret recording, harassment and extortion. 

The coming conflict over face cams

On one hand, all the biggest consumer electronics companies are either shipping AI glasses with cameras in them or planning to do so — and many smaller companies are looking to do the same. The industry expects AI glasses with cameras to go totally mainstream. 

On the other hand, a growing public, legal and legislative backlash has erupted in opposition to AI glasses with cameras in them. 

One possible outcome is that the public disdain for the cameras will fade, overwhelmed by widespread enthusiasm for the benefits they offer. A new social norm might emerge that mirrors the broad acceptance of everybody having cameras in their phones and pointing them in random directions. 

Another possibility is that companies will be forced by consumer disdain and legal action to abandon cameras in glasses and focus instead on AI glasses that can’t take pictures or use video for multimodal AI input. 

Either way, the war is surely coming.