Apple:

Audio Intelligence is a brand-new class of Apple Intelligence features that use the microphones of Apple Watch, the power of Apple silicon, and on-device and cloud-based AI models to help you detect sounds, identify music, and stay present and engaged — all while protecting your privacy at every step.

Hartley Charlton:

• Siri Recap: Uses ambient listening to generate high-level notes from conversations, such as takeaways from a team meeting or a parent-teacher conference, producing an Apple Intelligence-generated title, summary, and key points reviewable in the Siri app.
• Live Rewind: Lets users “go back in time for 15 seconds,” transcribing and displaying what was just said in a conversation with a double press of the Digital Crown.
• Shazam: Automatically identifies music playing nearby and surfaces the song title and artist on the Music Recognition widget in the Smart Stack.
• Sound Recognition: Brings sound alerts to Apple Watch for users who are deaf or hard of hearing, listening for sirens, alarms, doorbells, a baby crying, and more, even when their iPhone isn’t nearby.

Lily Hay Newman:

Apple is emphasizing that all of them are built to prioritize privacy and security using the extensive infrastructure the company has developed to protect its other Apple Intelligence and Siri services. As more and more of these types of features debut, though, the audio intelligence announcements are a reminder that AI-powered tools are becoming increasingly universal in all areas of computing and life.

“These features do not create or store audio recordings, and raw audio used for processing is completely inaccessible to the operating systems, apps, the user, or Apple,” the company wrote in a report shared with WIRED.

[…]

Apple emphasizes that its on-device capabilities for Apple watches have expanded thanks to the company’s new S11 chips. The chips include a special memory-protected space that Apple calls the Secure Exclave. Designed specifically for sensor data, the exclave is an isolated buffer that is inaccessible to the rest of the operating system where Apple Watch can hold and process audio data in a protected space.

[…]

To try to mitigate blatant privacy violations, Apple says that the watch produces an audible chime when a watch owner initiates a Live Rewind transcript to alert other people in the area—even if it is on silent mode or paired with headphones. And the features are built to omit or filter out potentially identifying information about speakers. But the sheer scope and breadth of the tools is extremely noteworthy.

Zac Hall:

If you want to really take a deep dive on privacy and the new features, Apple has published an 11-page privacy paper that goes even further.

Previously:

• Apple Watch Ultra 4
• Apple Watch Series 12
• Pebble Index

Update (2026-09-11): Dr. Drang:

Ron Huang introduced a couple of features of the Series 12 and Ultra 4 that struck me as both disturbing and possibly illegal. They certainly seem at odds with Apple’s commitment to privacy.

[…]

I’m sure Apple will argue that there’s no privacy issue because the recordings aren’t being saved. But if there’s a transcription, isn’t that a type of saving?

And then there’s the question of legality. A great variety of law, both in the United States and across the world, concerns the recording of electronic and in-person conversations. I’m sure Apple has considered this, but I have to think some jurisdictions will frown on Live Rewind. If Apple has to defend it, that’s fine. What’s one more squadron of lawyers to Apple? But if an Apple Watch owner has to defend a transcription made innocently—made with Apple’s encouragement, in fact—that seems like an unfair burden. Maybe Apple should indemnify all future Watch buyers from privacy-related legal action.

[…]

The hoops Apple jumped through to avoid certain words is pretty funny. […] You think your watch records the audio around you? No no no. The audio flows into the Secure Enclave, and flowing is definitely not recording because it’s a different word.

Juli Clover:

Live Rewind and other Audio Intelligence features do not generate audio recordings. Audio flows into the Secure Exclave, is processed, and then deleted.

According to Apple, Live Rewind text snippets “disappear permanently” from the Apple Watch approximately 30 seconds after the Apple Watch display dims, unless the info is saved to the Siri app.

Nick Heer:

But good luck explaining that to anyone around you after you read back an exact transcript of what they just said. Apple says they should be sufficiently notified Live Rewind is active because “an audible chime plays from the speaker on your Apple Watch, even if your Apple Watch is on silent or you have headphones connected”, plus the watch plays an animation, but it is not clear to me when this happens. It is not shown in Apple’s video. The impression I get is that Live Rewind is always running in the background and these notifications are only played after you double-click the Digital Crown to look at the text.

This feels like a glimpse of how Apple will market its inevitable wear-anywhere glasses product. I do not think it does enough to assuage privacy concerns. I think normal people will continue to react negatively when you tell them you have been passively recording them because, outside Silicon Valley, that is considered gross and invasive.

Saagar Jha:

I have to admit that the Siri Recap stuff all being “documented” using AI slop is really giving me confidence in the level of care that went into designing this feature

Matt Evans (Hacker News):

I’m frankly surprised Apple even went through with this feature at this time, considering the bans and backlash other AI-powered surveillance wearables are getting.

[…]

Apple is keen to evidence its trustworthiness, with on-device AI and Private Cloud Compute and an insistence that it doesn’t record or log audio and transcripts. However, if people think they’re being discreetly recorded without their permission, it’s possible Apple Watch wearers could experience their own slice of the abuse hurled at Meta Glasses wearers.

Dan Sugalski:

I… y’know, Betteridge’s law does, as always, apply.

Glenn Fleishman:

Given that Apple has to warn of this, it’s the surest sign the features shouldn’t be released in this form. Apple repeats the form of the warning I see regularly at my gym… but apparently it isn’t heeding it.

Jack Brewster:

Just called my HSA provider and upfront they say that calls may be recorded and transcribed by AI and then provide no means of opting out. So of course the person who finally answered says I have to give my social to verify my identity.

Update (2026-09-14): See also: Core Intuition and Steven Aquino.

Om Malik (2023):

One of the key advantages of MacWhisper, as it utilizes the Whisper technology, is its superior accuracy in transcriptions compared to other services I have used. I was an early adopter of Rev.com and eventually switched to Otter.ai, for which I paid approximately $100/year. However, I have since canceled my subscription.

[…]

After comparing the audio transcripts from MacWhisper and Otter, I found that while Otter’s formatting is superior, there is hardly any difference in the transcriptions. I started with the free model, which was sufficient for my needs, and eventually upgraded to the large models. Nowadays, I primarily use MacWhisper to transcribe my voice memos, which I then import into Lex.Page.

Bon Adamson:

Whisper, which is made by OpenAI, has the distinct advantage of allowing you to transcribe audio in almost any language you may ever need to. For the time being, Otter can only transcribe English. The benefits of better accuracy and accurate punctuation that the service already holds over Siri hold up across other languages as well. If you’re looking to transcribe non-English audio accurately, Whisper is definitely the best option available on Mac.

Dan Moren:

By taking advantage of OpenAI’s Whisper framework, Audio Hijack can now take any audio it’s recording and generate a text transcript. While this process was possible before—see Jason’s lengthy post about his workflow—it required a lot of fiddling and several different tools to accomplish; it’s certainly a lot easier to simply drop Audio Hijack’s block into your existing session. If you record multiple sources, it can even appropriately label each one—though if you record multiple people via one input, say the other end of a Zoom call, it can’t distinguish between the various participants.

Inscribe (2026, Hacker News):

Apple’s new SpeechAnalyzer is the most accurate on-device speech engine we tested. It beat every Whisper model we ship, including Whisper Small, on both the clean and the noisy half of LibriSpeech, while running roughly three times faster than Small. And the API it replaces, SFSpeechRecognizer, came last on clean speech: behind even Whisper Tiny, a 40MB model.

[…]

The new API cuts word error rate by 3.5 to 4x on the same audio: from 9.02% to 2.12% on clean speech, and from 16.25% to 4.56% on noisy speech. There is no accuracy trade-off to weigh; the new API wins everywhere we measured, and it produces punctuated, cased text where the legacy engine’s output is rougher.

[…]

All five engines ran comfortably faster than real time: between roughly 12x and 40x on the M2 Pro, meaning an hour of audio transcribes in about 1.5 to 5 minutes on-device. SpeechAnalyzer was about 3x faster than Whisper Small per second of audio while beating it on accuracy.

Yuki:

SpeechAnalyzer’s ReportingOption.fastResults reads like a knob you’d reach for when you want to trade power for latency. That framing is what cost me the afternoon.

[…]

If a recording matters — and in a notes app, a lost meeting is the worst thing that can happen to a user — then the container you write while recording has to be PCM in CAF. Not m4a, and not AAC-in-CAF either.

[…]

AVAudioEngine’s inputNode silently ignores your device selection

[…]

I wanted summarization to run on-device. Everything else in Nib’s transcription path does, the audio never leaves the machine, and it would have been a much cleaner story to tell.

But 4,096 tokens for the Foundation Models isn’t enough.

Previously:

• Apple Foundation Models in appleOS 27
• WhisperPad Rejected From the Mac App Store
• Overcast Transcripts
• Generated Transcripts in Pocket Casts
• Apple Podcasts Transcripts
• Dictation and Predictive Text in macOS Sonoma and iOS 17
• macOS vs. Dragon Dictation