❌

Reading view

SuperDuper’s “Follow-Up”

Dave Nanian:

Recent versions of xOS (that is, macOS, iOS, iPadOS, etc) have Data Protection Classes from A to D that get assigned to files and folders.

[…]

The thing is, though, is that these files and folders are inaccessible unless the user is logged in, and the screen is unlocked. A few seconds after the screen locks (or you log out), the keys are purged, and they cannot be read.

[…]

That means the files any application considers to be worth protection cannot be accessed when a backup usually happens.

[…]

You may have noticed that if you had an overnight backup, when you unlock your screen, a copy starts to run. It’s typically quite brief, thanks to ⚡️Turbo Smart Update, but it might surprise you anyway. […] During your overnight copy, SuperDuper kept track of the files that it couldn’t access due to their protection class. And now that it can access them, it’s making sure they get backed up.

This seems like the right thing to do. Time Machine also does it. A backup app should try to make as faithful a backup as possible. I kind of find it annoying, though. Sometimes it can’t do a Turbo Smart Update, and so the backup that should already be done may end up taking another 30 minutes or more. The best thing to do is schedule backups while you’re using your Mac to avoid SuperDuper needing to do such “follow-ups.”

I wrote some code to try to figure out which files I have that are using data protection. Interestingly, the URLFileProtection API seems to not work on macOS. I used the lower level getattrlist() instead. Nanian recommends fcntl(). I didn’t find any protected files that I actually care about, just caches and derived data from Apple’s own apps. But, in theory, Apple or a third-party app could be saving actual user data using data protection.

Previously:

  •  

Aren’t Snapshots Purgeable?

Howard Oakley:

Since High Sierra and APFS snapshots, accounting for purgeable space has been inconsistent and often misleading. I first reported this over eight years ago in Sierra 10.12.6 and High Sierra 10.13.3, and most recently three years ago in Ventura 13.3.1. This article looks at how this is handled now in Tahoe 26.6.2.

[…]

There is disappointingly little documentation on space management in macOS, and the closest Apple comes to defining purgeable space is in Disk Utility’s Help pages, where it’s stated to be “space that macOS can free up when needed by removing files from your computer (you can’t manually remove the files that are designated purgeable, but macOS removes them as space is required)”.

On the face of it, that includes cache files but not snapshots, as they can be manually removed in Disk Utility, as I’ve discussed in detail here. This is contradicted when using tmutil localsnapshot to make Time Machine snapshots, which reports that “local snapshots are considered purgeable and may be removed at any time by deleted(8)”, as deleted is the purgeable space and cache management service in macOS.

APFS snapshots can also be created by third-party apps that have the appropriate restricted entitlement. Apple doesn’t explain whether those too can be purged by macOS when additional free space is required.

Howard Oakley:

Before macOS will start copying a file to a volume, it checks whether there’s sufficient free space available. I believe this is normally performed by the FileCoordinator API in Foundation, and appears in the log with the subsystem com.apple.foundation.filecoordination.

If there isn’t sufficient free space to accommodate the file to be copied, deleted in the subsystem com.apple.cache_delete checks whether enough space can be freed by purging items known to be purgeable in the destination volume. deleted has a fixed set of services that can be called on to purge items, listed in /System/Library/CacheDelete.

[…]

Space-based purging of Time Machine local snapshots is unreliable. The reason for this is that Time Machine CacheDelete fails to report purgeable space when it’s available. Even when it does work, it can take nearly half an hour for Time Machine CacheDelete to recognise the appearance of purgeable space.

Sometimes you need to do it yourself with Disk Utility.

Previously:

Update (2026-09-03): Howard Oakley:

  • When purgeable space is required, deleted (subsystem com.apple.cache_delete) first discovers purgeable space for each caching service, to determine whether it’s sufficient.
  • If it is, and the decision is taken to purge caches, caching services are called on to free up space by purging their caches as required.
  • Purging may then take place in phases until sufficient space has been freed.
  • Purging is frequent during installation of macOS updates, even when there’s ample free space available.

Update (2026-09-08): Howard Oakley:

In the event that you’re caught out by circumstances, such as the combination of recent large snapshots and exceptionally large caches, macOS does try to manage as well as possible even when there’s just a few GB free. But once free space falls to around 1 GB you should see a warning notification.

[…]

From then on, most file operations, even creating a new folder, will fail. That applies to all apps, regardless of whether they handle those failures gracefully and warn you of the error. Actions like taking a screenshot start normally, but after their image has floated across the display, they just vanish and can’t be saved.

[…]

The worst combination of problems occurs when a Mac runs out of free disk space and physical memory at the same time. macOS memory management then wants to allocate space in the VM volume as its backing store for virtual memory, but in the absence of free space to do so, that fails and should propagate as a memory allocation error.

Update (2026-09-09): See also: Accidental Tech Podcast.

  •  

Sunsetting CoreStorage

Apple:

In macOS 28 and later, the Mac OS Extended file system format will be supported only for volumes (disks and other storage devices) that aren’t encrypted. For future macOS compatibility, either decrypt or reformat any encrypted Mac OS Extended volumes.

I think Apple removed the user interface for creating such volumes way back in Big Sur. It’s not really surprising that they’re going away, although only offering a year of warnings seems rather aggressive. Many people delay updating to major OS versions or wait until they get a new Mac. There will be users that switch to a Mac that can only run macOS 28 and get rid of their old one before realizing that some drive they haven’t connected in a while won’t mount.

The other surprising part is that Apple’s options—reformat or decrypt—are not great. During the initial transition to APFS, macOS would transparently convert an encrypted HFS+ boot volume to encrypted APFS without rewriting any data. This functionality is nominally still available via the diskutil apfs convert command, but Apple hasn’t mentioned this or built an automatic upgrader GUI.

[Decrypting] doesn’t apply to encrypted Time Machine backup disks.

It’s obvious why. The old Time Machine format with hard links is totally different from the new one based on APFS snapshots. But this means that you’ll either need to buy a new Time Machine drive for use with macOS 28 or delete your backup history. Even if you keep the old drive, you’ll need an older boot drive or an old Mac in order to be able to access it.

Previously:

Update (2026-07-24): Marcus Mendes:

Apple has now revised the page to add a third option, allowing users to convert an encrypted HFS+ volume directly to encrypted APFS without decrypting it first.

[…]

The company has also removed its previous instructions for decrypting the volume first, converting it to APFS without erasing it, and then optionally re-encrypting it.

Hammer:

Ignore Apple’s guidance and do not use Convert to APFS in Disk Utility for disk images.

The first encrypted HFS+ disk image I tried to convert that way failed and left me with an unusable disk image. If I didn’t have backups I’d have lost its contents.

It didn’t corrupt the disk images that I tried, but it didn’t work, either, instead failing with an error about expecting a GPT partitioning scheme.

Update (2026-07-28): Howard Oakley:

As encrypted disk images have never used CoreStorage, withdrawal of support for that has no impact on the continued use of any form of encrypted disk image, including those using HFS+ as their virtual file system.

This is a good point. Apple’s support page refers to “encrypted Mac OS Extended disks,” but the developer release notes refer to CoreStorage specifically. So this shouldn’t affect most encrypted disk images, which show up in Finder as unencrypted after mounting. The encryption is at the disk image level rather than at the file system level. However, I’m not sure that all encrypted disk images are safe. Presumably, a device image, which directly copies the blocks rather than the files, would still use CoreStorage.

  •  

Searching Time Machine Backups and Versions

Howard Oakley:

Spotlight doesn’t currently appear able to search Time Machine backups reliably, at least not in Sequoia or Tahoe, although this may not be universal.

[…]

If I now step back through my backups to reach one that I know contains that file, I can restore it. But if I type anything into the search field, nothing is found. If I change the scope of the search to that backup, the window title changes but its contents remain blank, and there isn’t even a busy spinner to indicate a search is in progress.

[…]

As I can’t disable Spotlight indexing on that volume without macOS telling me that it’s required to be indexed by Spotlight, neither can I force that volume to be reindexed.

[…]

My conclusion is that using Spotlight to search Time Machine backups no longer works, and the instructions given by Apple are also broken.

That has been my experience, too. As I see it, there are four separate problems:

  1. I want to be able to search all my backup snapshots, because I don’t know which one might contain the file I want to restore. Time Machine only lets you search one at a time.
  2. Searching an individual snapshot doesn’t actually work.
  3. Spotlight indexing is invasive, using lots of CPU time, and with spinning hard drives causes wear and tear and creates lots of noise. If search doesn’t work anyway, I’d like to turn it off, but macOS won’t let me.
  4. If the problem is with the index, there’s no way to reset it without deleting your backup.

Howard Oakley:

Unfortunately, all those saved versions in the version database fall outside the scope of Spotlight indexing, and Spotlight search can’t look inside any of the old versions saved in a volume’s version database. Surprisingly, the version browser doesn’t offer any search facilities either, as that’s presumably another feature intended for a future that never came.

[…]

One way around this is to save each document version as a separate file, allow Spotlight to extract their contents and add those to its indexes for that volume, then to search those files. This is quick and simple using my free utility Versatility.

Previously:

  •  
❌