❌

Normal view

Whisper and SpeechAnalyzer

22 July 2026 at 21:34

Om Malik (2023):

One of the key advantages of MacWhisper, as it utilizes the Whisper technology, is its superior accuracy in transcriptions compared to other services I have used. I was an early adopter of Rev.com and eventually switched to Otter.ai, for which I paid approximately $100/year. However, I have since canceled my subscription.

[…]

After comparing the audio transcripts from MacWhisper and Otter, I found that while Otter’s formatting is superior, there is hardly any difference in the transcriptions. I started with the free model, which was sufficient for my needs, and eventually upgraded to the large models. Nowadays, I primarily use MacWhisper to transcribe my voice memos, which I then import into Lex.Page.

Bon Adamson:

Whisper, which is made by OpenAI, has the distinct advantage of allowing you to transcribe audio in almost any language you may ever need to. For the time being, Otter can only transcribe English. The benefits of better accuracy and accurate punctuation that the service already holds over Siri hold up across other languages as well. If you’re looking to transcribe non-English audio accurately, Whisper is definitely the best option available on Mac.

Dan Moren:

By taking advantage of OpenAI’s Whisper framework, Audio Hijack can now take any audio it’s recording and generate a text transcript. While this process was possible before—see Jason’s lengthy post about his workflow—it required a lot of fiddling and several different tools to accomplish; it’s certainly a lot easier to simply drop Audio Hijack’s block into your existing session. If you record multiple sources, it can even appropriately label each one—though if you record multiple people via one input, say the other end of a Zoom call, it can’t distinguish between the various participants.

Inscribe (2026, Hacker News):

Apple’s new SpeechAnalyzer is the most accurate on-device speech engine we tested. It beat every Whisper model we ship, including Whisper Small, on both the clean and the noisy half of LibriSpeech, while running roughly three times faster than Small. And the API it replaces, SFSpeechRecognizer, came last on clean speech: behind even Whisper Tiny, a 40MB model.

[…]

The new API cuts word error rate by 3.5 to 4x on the same audio: from 9.02% to 2.12% on clean speech, and from 16.25% to 4.56% on noisy speech. There is no accuracy trade-off to weigh; the new API wins everywhere we measured, and it produces punctuated, cased text where the legacy engine’s output is rougher.

[…]

All five engines ran comfortably faster than real time: between roughly 12x and 40x on the M2 Pro, meaning an hour of audio transcribes in about 1.5 to 5 minutes on-device. SpeechAnalyzer was about 3x faster than Whisper Small per second of audio while beating it on accuracy.

Yuki:

SpeechAnalyzer’s ReportingOption.fastResults reads like a knob you’d reach for when you want to trade power for latency. That framing is what cost me the afternoon.

[…]

If a recording matters — and in a notes app, a lost meeting is the worst thing that can happen to a user — then the container you write while recording has to be PCM in CAF. Not m4a, and not AAC-in-CAF either.

[…]

AVAudioEngine’s inputNode silently ignores your device selection

[…]

I wanted summarization to run on-device. Everything else in Nib’s transcription path does, the audio never leaves the machine, and it would have been a much cleaner story to tell.

But 4,096 tokens for the Foundation Models isn’t enough.

Previously:

Apple Foundation Models in appleOS 27

16 June 2026 at 20:35

Apple:

At the heart of this architecture is our third generation of Apple Foundation Models (AFM), a family of five foundation models custom-built in collaboration with Google. These span from on-device models to server-based models running on Private Cloud Compute.

Apple Foundation Models are built to unlock a wide range of helpful experiences for our users, like an entirely new Siri and intelligent tools that make everyday apps smarter and more useful.

Hartley Charlton:

“The amount of the Google Assistant we use is none,” Federighi said, explaining that Apple uses none of the Gemini models deployed to Google’s customers, none of Google’s client-side code, and no Google Search infrastructure as the knowledge backbone.

Federico Viticci:

The cloud models hosted on Private Cloud Compute are interesting: they’re not “Gemini” models; they’re Apple Foundation Models trained using proprietary data via RL, which were then “refined” with data from Google’s “frontier” models.

[…]

I’ve been covering the AFM family of models for a while now, and AFM Core Advanced is one of the most interesting on-device models I’ve read about in a while, especially in the context of model size for mobile devices and built-in multimodality with support for text, images, and audio. I’m very keen to play around with this model and understand how it holds up in practice. I wonder if the new CLI (!) for AFM will let you test this one.

Awni Hannun:

It’s very cool that Apple shipped a 20B parameter on-device.

You can’t put 20B parameters in RAM at any reasonable precision. To make it work they are using pretty exotic architecture by today’s standards.

A small model predicts from the query (or prompt) which experts to load from Nand into RAM. The key distinction from a typical MoE is that you do this once per query and then generate all the tokens with the same experts (instead of switching the experts for every token).

Greg Pierce:

Looks like there’s still a 4K context size on local FoundationModels. Pretty limiting.

Kyle Howells:

Really disappointed SwiftUI’s result builder syntax has infected FoundationModels API too.

How did a million modifier methods become the default way to build APIs?

It’s horrible for discoverability, code completion, documentation.

Steve Troughton-Smith:

FoundationModels is one of the highlights of this WWDC; I think it’s now finally what we hoped we were getting two years ago, and thought we were getting last year (just the models themselves were barely up for it then). I finally feel like I can build features with it, though I really don’t like the Sword of Damocles Apple is holding above Private Cloud Compute and how it can permanently lock you out of using the models forever across all your apps if you hit a download threshold. Poison pill

John Gruber (Mastodon):

These strict limits don’t seem to be getting as much attention as they should. It’s nice that for small developers who meet the above criteria, access to PCC has no cost. But there’s no way (yet?) to buy your way out of these limits.

[…]

The “fewer than 2 million first-time app downloads from any of their apps” restriction is particularly notable. It’s not 2 million installations for apps that are using PCC, but 2 million downloads for any app the developer has ever released.

Chad Podoski:

Is there a real time count somewhere of total lifetime downloads for a developer account, so one can plan if they at risk of the switch tripping? Reminds me of the small business qualifications. I had to try to figure out if an employers developer account was going to exceed the revenue threshold for the year. I was left trying to determine it from sales reports, not to mention sales date versus pay out date confusion. I think I botched the calculation in the end.

Anthropic (Hacker News):

Claude for Foundation Models is a Swift package that makes Claude available as a server-side language model in Apple’s Foundation Models framework. The package conforms Claude to the framework’s LanguageModel protocol, so you drive it with the same LanguageModelSession API you use for Apple’s on-device model: respond(to:), streaming, guided generation, and tool calling all work the same way.

Requests go directly from your app to the Claude API; Apple is not in the request path and does not see prompts or responses. Usage is billed to your Anthropic account at standard API pricing.

Previously:

Update (2026-06-17): Simon B. Støvring:

Apple is literally stopping me from building a brand new app using Private Cloud Compute because I made a successful app a few years ago.

The new app might be a success, or it might be a complete flop. Nobody knows yet.

I would understand it if Apple were to limit developers once an app reaches a certain number of downloads or active users, but that’s not what’s happening here.

Instead, they’re blocking developers before they’ve even had a chance to build and ship the app.

Siri AI Announced

10 June 2026 at 21:00

Apple (Hacker News):

Siri AI is an entirely new version of Siri deeply integrated into iPhone, iPad, Mac, Apple Watch, and Apple Vision Pro. It can draw on personal context understanding to search across messages, emails, photos, and more, and get things done across apps with even more systemwide app actions. Additionally, Siri AI can answer questions related to the content on a user’s screen or go out to the web to get up-to-date information using broad world knowledge and generate a helpful answer. A dedicated Siri app allows users to revisit a past conversation or kick off a new one — all in one place — and uses iCloud to privately sync conversational history across a user’s products.

Apple (MacRumors, MacStories):

Powered by the next generation of Apple Intelligence, Siri AI is a completely reimagined version of Siri that is more helpful, more capable, and more intelligent. With detailed, engaging responses and natural back-and-forth conversation, Siri AI helps users get more done than ever.

[…]

Siri has been rebuilt from the ground up with powerful AI at its core. It takes full advantage of the bold new architecture for Apple Intelligence, including the next generation of Apple Foundation Models that run on device and on servers using Private Cloud Compute. When Private Cloud Compute is handling users’ requests, their personal data is not stored nor made accessible to Apple or anyone else. Outside experts can continue to verify this privacy promise at any time. Additionally, Siri AI uses the system orchestrator to tap into core capabilities like the Spotlight index and App Toolbox, which work entirely on device to keep users in control of their data.

[…]

For products that support Apple’s most advanced on-device model ever, Siri AI offers even more expressive voices, as well as a major boost in accuracy with systemwide dictation. Users have the ability to customize the expressiveness and pace of Siri’s voice so it’s just right for them. Dictation now captures what users say as polished text with greater precision, automatically handling capitalization, punctuation, and formatting as they speak. With improved speech understanding, users can speak naturally and trust that their words will appear clearly, accurately, and as intended.

I remain uninterested in these more advanced Siri features. They matter in that Apple needs to show that they’re not “behind” and that they can actually deliver on what they announce and advertise. But I don’t find the use cases compelling and wouldn’t trust the results.

If you really care about a particular concert, would you believe Siri about how the tickets work and that it will auto-create a reminder at the right time? Why should I believe that it can figure out whether a particular irregularly shaped object can fit inside an irregularly shaped container when said shapes are probably not even published on the Web? Can it really dig into my e-mail to find the right flight info? An answer without being able to see the sources is not useful. And I certainly don’t want it to pick for itself which vacation photos to send out.

I just want the basic features—from a decade or so ago, like music controls, reminders, and access to my purchases—to work quickly, reliably, and offline to the extent possible. I did not hear anything to suggest that this was a focus. Instead, everyone is celebrating how slow the demos were because that proves they weren’t fake.

Ben Thompson:

It’s actually super important that the Siri demo was kind of slow, because it emphasized it was real. Would be even better if it were live, though.

Mark Gurman:

The other interesting thing to note from the keynote: watching the demos, there is ZERO doubt these are real, recorded demonstrations on live devices. There will be no confusion about what is real or fake like there was at WWDC 2024 and the initial attempt at the new Siri.

John Gruber:

In fact, I think most of them were single takes of multiple demos back-to-back. That’s the way it should be, even when they feel a little slow.

Jonathan Wight:

Who the fuck wears a $4000 VR headset to find out if their boots will fit in their fucking backpack?

THATS the best example they could come up for that feature?

Rob Terrell:

also I wouldn’t trust the answer anyway

Kyle Howells:

These Siri demo’s feel like OpenAI demo’s from a view years ago. However, this is built in to the operating system.

Much like Dropbox vs iCloud, I feel like long term, once it’s good enough Apple will succeed in making AI a “feature” rather than a product.

Simon B. Støvring:

Apple needed to show that they could make a version of Siri that wasn’t three years behind. They didn’t succeed.

Ben Thompson (Hacker News):

And, if your standards are the state of the art in AI circa June 2024, when Apple took their first crack at answering the question, they did quite well.

[…]

What’s fascinating about this specific demo is that it also showed just how far behind Apple is.

[…]

Still, the bubble was strong enough to convince OpenAI to make the exact same mistake Dropbox did: the company somehow convinced itself that it could make enough money selling subscriptions to consumers; Anthropic, meanwhile, realized that it was enterprises who were willing to pay for AI’s massive productivity benefits, even as OpenAI failed to capitalize on their consumer market penetration by refusing to build an advertising product.

This is a long-winded way of saying that I don’t think that Apple’s agentic shortcomings are a big deal, at least for now. Agents help you do work and be more productive, and consumers don’t want to work or care about being productive. What they do want to do is watch short-form video, and an iPhone is simply much better at that than any other device ever will be; in that context, Siri being good enough is enough, and it appears that Apple crossed that bar.

John Gruber:

They need to execute, they need to prove this can scale, and most of all, they need to get third-party apps on board with App Intents and App Schemas. But it seems like they’re doing all of that. This is not a done deal but it is very realistic.

Steve Troughton-Smith:

Siri doesn’t have access to the filesystem or tool-calling, so it’s completely unable to do the kinds of tasks you might easily do with Codex or Claude. It’s a good start, but the state of the art has already dramatically shifted over the past year. The question is will Siri move in this direction, or stick to providing answers from the web?

Will Apple introduce its own distinct Codex-style agentic programming model that can do all this, and work with Xcode? You would hope so…

Aaron Pearce:

Have new Siri now. Siri still can’t do “turn lamp off and turn plug on” in one command.

Joe Rossignol:

The revamped version of Siri that is officially named “Siri AI” will be available to test in the iOS 27, iPadOS 27, macOS 27, and visionOS 27 developer betas starting today, and it is coming to the Apple Watch in a future watchOS 27 beta. “Siri AI” will also be included in the public betas of each platform that are launching in July.

“Siri AI” requires a device that is compatible with Apple Intelligence.

Hartley Charlton:

Apple senior vice president of software engineering Craig Federighi has explained why the company launched a standalone Siri app in iOS 27, after previously characterizing a dedicated chatbot as contrary to its Apple Intelligence strategy.

[…]

“We see Siri not as a separate chatbot, just an unintegrated place you go and chit-chat, but rather as an integral, conversational tool that you use in the moment, deeply integrated into your experience.”

Clear as mud.

ldt:

How to bypass the new Siri waitlist (Mac only)

See also: New Siri Waitlist.

Previously:

Update (2026-06-11): famebot:

…And playlists, contacts, settings and preferences. Peel off the agentic slop bandwagon and focus on core performance, smarter caching, and resilience under adverse network conditions.

Joe Fabisevich:

Love the new Siri, sick.

Update (2026-06-12): Juli Clover has a hands-on video.

Nick Heer:

Though this is a test conducted on the very first version of Siri A.I. from a fixed point in Calgary, it seems to work quite well. So far, it is better than any version of Siri Apple has released yet and, as you can see above, it is almost as good as the original 2010 demo, before Apple acquired the company.

Update (2026-06-16): Riccardo Mori:

On the one hand, if Siri has indeed become this reliable at understanding queries like those demoed so far, it’s indeed impressive. On the other hand, I can’t help but ask myself, once again, whether people at Apple even know how regular people actually use their phones and devices.

Jeff Johnson:

Apple executives downplay and obfuscate the role of Google in Siri AI. The Siri AI announcement didn’t even mention Google or Gemini once! Does anyone actually believe Apple’s BS? The argument seems to be that since Google made a bespoke implementation for Apple, Google is not really involved, and it’s all Apple. Ridiculous.

Jeff Johnson:

It’s really odd to me that Apple is getting so much credit for Google’s work. Siri now “works” to the extent it does because Siri is to a very large extent Gemini, and Apple has wrapped that in a bow.

Apple didn’t “nail” anything. Apple failed at its own internal efforts and was forced to outsource.

This is the advantage of being a platform: you can fail and still win.

Chance Miller (via John Gruber):

Apple’s Siri team, led by Craig Federighi, held a post-WWDC keynote tech talk with members of the press this afternoon to talk through iOS 27 and the new Siri AI.

During the talk, Federighi shared more details about Apple’s collaboration with Google.

Matthew Green (Lobsters):

Apple looks like it will use some combination of Google Gemini models, combined with Google’s Confidential Inference and Apple’s own Private Cloud Compute for private hosting. These systems will process both your queries and evaluate private data from your devices.

[…]

Apple has since “expanded” PCC to encompass Google’s hardware as well. I will confess that I find the details of the new “expanded” PCC just a bit vague. It sounds a lot like Apple is primarily going to rely on Google’s existing confidential compute (running in Google datacenters) to process this data, but they’re bolting on a new layer of technical security to control which models are actually running. In any case: security experts can argue about whether this is good enough to keep Cozy Bear away from your data. What I will grant is that it’s probably good enough to keep Google and Apple from accessing your stuff, which is what most people are worried about in the first place.

So why am I so nervous?

[…]

So far we’ve discussed two adversaries: the misaligned designers of private agents (such as search operators), and the possibility of remote prompt injection attacks. But of course, in any discussion of technical privacy systems we need to talk about the last elephant in the room: your government.

Previously:

“Apple Upgrade”: empresa poderá bloquear apps por pagamentos atrasados

22 July 2026 at 13:32

Ontem, falamos aqui sobre o rumor de que a Apple estaria prestes a lançar, nos Estados Unidos, um programa de assinatura de iPhones, iPads, Macs e Apple Watches, dando aos consumidores mais formas de acessar seus dispositivos.

Logo depois, o 9to5Mac encontrou, no código da beta do iOS 27, menções a um novo framework que serviria para gerenciar dispositivos alugados por meio de assinaturas como essa.

Chamado App Managed Features, ele permitirá, ao que tudo indica, que um aplicativo autorizado de financiamento ou de prestação de serviços — e não apenas o “Apple Upgrade”, como a solução proprietária está sendo chamada — registre um iPhone e realize verificações contínuas de status.

Caso haja algum problema com o registro (como pagamentos atrasados), o iPhone será colocado automaticamente em um estado intitulado “Restricted Mode” (“Modo Restrito”, em tradução livre), que bloqueia o acesso a uma série de apps do iOS, incluindo:

  • Leitor de Acessibilidade (Accessibility Reader)
  • App Store
  • Saúde (Health)
  • Lupa (Magnifier)
  • Telefone (Phone)
  • Relógio (Clock)
  • Ajustes (Settings)
  • Carteira (Wallet)
  • Senhas (Passwords)
  • A interface do “Restricted Mode” em si

Como é possível notar, alguns apps mais “críticos”, como o Mensagens (Messages) e o Casa (Home), poderão continuar disponíveis mesmo quando esse bloqueio estiver ativo. Segundo o 9to5Mac, no entanto, o provedor do aparelho será capaz de escolher quais apps ficarão (ou não) de fora dessa lista.

Embora restrinja o acesso aos apps do aparelho, o recurso não parece modificar assinaturas da App Store, o que significa que o usuário continuará sendo cobrado pelos seus apps e serviços mesmo após ter sido colocado no “Restricted Mode”.

Além disso, o código do iOS 27 não lista um número limite de pagamentos atrasados como condição para o bloqueio dos apps, o que significa que ficará a cargo dos serviços de assinatura decidir quando impor essas restrições de acordo com os seus termos e condições.

Há, no entanto, menções a um novo tipo de Bloqueio de Ativação (Activation Lock) chamado “Partner Finance Lock”, criado com o objetivo de impedir que usuários apaguem, vendam ou desmontem um aparelho alugado para revender suas peças.

Ainda segundo as informações, o App Managed Features faz parte do app Buscar (Find My), embora o provedor não tenha acesso à localização do dispositivo. Ao que tudo indica, essa integração existe apenas para garantir que o bloqueio continue de pé mesmo que o usuário apague ou redefina o aparelho.

Embora essas referências não tenham sido encontradas na versão Release Candidate (RC) do iOS 26.6, é possível que elas sejam adicionadas à versão final do sistema, já que, segundo a Bloomberg, o “Apple Upgrade” será anunciado na semana que vem.

❌