EagleFiler 1.9.22 is a maintenance release of my Mac file/note organizer and e-mail archiving app. This version works around various bugs in macOS 27, adds capture support for the Thorium Web browser, and improves importing from Mariner Paperless.

• macOS 27 broke several things with drag and drop out of Apple Mail. Simply trying to read a URL from the pasteboard will cause a hang while it waits forever for the pasteboard server (FB24803817). The same thing happens when using PasteboardSetPasteLocation() and reading kPasteboardTypeFileURLPromise directly, which I was doing (instead of using newer API) to work around bugs in older versions of macOS. EagleFiler now uses NSFilePromiseReceiver for drags from Mail on macOS 27, which works better, but I also had to work around a bug where Mail would promise superfluous files for which the reader callback was never invoked at all.

• macOS 27 also seems to include various regressions when reading PDF and Word files, producing invalid strings containing NUL characters. Also, displaying certain PDF files worked fine on macOS 26 but now crashes.

• I also ran into (and worked around) a bug where macOS 27’s NSTextView would get into an infinite layout loop when displaying an attributed string that included images.

• Drag and drop from Airmail is still an issue. When using the newer drag APIs, it will often hang Airmail or not produce a file, and this also happens with other apps like Finder as the destination. EagleFiler works around this using older APIs with Airmail and tries to report a helpful error when there’s trouble. Single-message drags work best.

• I’ve also been investigating a problem where saving a file with a colored tag in iCloud Drive will, a few seconds later, change the color to gray, but I don’t have a solution for this.

Previously:

• EagleFiler 1.9.21

Nicolas Seriot:

// "<a>" in UTF-16 LE + a single dangling byte
const UInt8 bytes[] = {'<', 0, 'a', 0, '>', 0, 'X'};

[…]

The whole bug is a confusion between bytes left versus UTF-16 characters left. The result is repeated buffer doubling until allocation fails, and a local process crash if the exception is uncaught.

Nicolas Seriot (Hacker News, Reddit):

The core Unicode algorithms (normalization, casing, bidi, collation) are deliberately bounded, but UTS #35 transliteration rules, under their natural unbounded semantics, are not. This is a result I haven’t found published before.

These rules ship as locale data in ICU, the Unicode library shipped in operating systems, browsers, runtimes, and databases. Whether a given rule file terminates on a given input is undecidable.

[…]

To prove UTS #35 universality, we compile 2-tag systems (Post, 1943), a model proven universal (Cocke & Minsky, 1964), into transliteration rules.

[…]

Transliteration rules were designed to turn “é” into “e”. Three lines of them can compute the Collatz function.

KT_Music:

Inspired by this, I tried creating an implementation that “solves #Sudoku” using only UTS#35 transformation rules 😎

Previously:

• Outlandish SQLite Recursive Query Examples
• Solving the “Miracle Sudoku” in Prolog
• Sudoku Solver

SpamSieve 3.3.1 is an update of my Mac e-mail spam filter that improves the filtering accuracy and includes various optimizations, fixes, and updates for macOS 27. Some interesting engineering issues were:

• I’m still supporting macOS 10.13, but the app can now build with the macOS 12 deployment target in preparation for Xcode 27. The way Swift handles calling deprecated APIs remains annoying. You need workarounds like protocols to suppress a warning in a particular region of code. Often, I am updating to a newer API, but I want to continue using the old one on earlier versions of macOS. Right now I am heavily testing on macOS 26, so I would like to only deploy the new code there. Sometimes the initial versions of APIs are buggy, and I don’t want to have to retest every feature on every version. But Swift doesn’t like me to gate the calls to macOS 26 and later if the new API was actually introduced in, say, macOS 11.

• The most risky update was probably switching to the Swift API for Scanner. This is tricky because the current scan position changes from being based on UTF-16 indexes to String.Index. And in some cases my code was intentionally scanning into the middle of a combining character. I was really glad to have extensive tests here.

• Recent versions of Tahoe introduced a Core Data bug that broke save recovery (when invalid Unicode can’t be written to the database and we have to fix it up first). When the error comes in, SpamSieve looks under the NSDetailedErrorsKey key, but the value of this key is "NSDetailedErrorsKey" while the actual error info is under the key "NSDetailedErrors".

• Database corruption combined with a Core Data bug and two Swift behaviors were causing crashes. The root problem, I think, is that in this situation Core Data neglects to set the error pointer for -[NSManagedObjectContext countForFetchRequest:error:]. Normally, Swift would detect this and throw a generic nilError. However, this method signals errors by returning NSNotFound instead of nil. It’s annotated as swift_error(nonnull_error), which I think means that Swift doesn’t even look at the return value, only the (unset) error. So my code was receiving NSNotFound on the non-error path, treating it as a real count, and doing some math with it, which (by default in Swift) leads to a crash if there’s overflow. Now my code checks for NSNotFound and uses Int.addingReportingOverflow().

• I reimplemented Core Data batch fetching because of two problems with the fetchBatchSize option. First, it logs spurious errors when fetching from multiple stores at once. Second, it leads to high memory use. It will load objects in batches, but it never releases the earlier batches. You can manually refault objects that you no longer need, but it will still hold onto the full object shells, which can add up. My implementation only holds onto IDs, and it can fetch the IDs using a different context than the objects to push more work to background threads.

• I’m now using String.makeContiguousUTF8() in various places to avoid slow iteration of bridged strings.

• There was a regression where some menu code could crash on macOS 10.13. -[NSMenu setItemArray:] is only available on macOS 10.14 and later, whereas the getter is always available. With old Objective-C, the two methods could be annotated with separate availabilities. The Objective-C property syntax doesn’t allow this, so there’s no way for the compiler to flag calls where the setter is missing.

• The AppleScript support works much better with large data sets. The key, I think, is to never return an array to Cocoa for a scripting property/element. First, if you give it a lazy array it will sometimes try to access all the elements even if it only needs a few. And, second, AppleScript will run out of memory if the array is too large. By using indexed accessors, you can maintain laziness and only convert one object to a descriptor at a time. By caching a batch-fetching collection, you can limit the number of fetches and respond quickly when AppleScript requests individual objects.

Previously:

• Xcode 27 Announced
• SpamSieve 3.3
• Swift Proposal: Precise Control Flags Over Compiler Warnings
• Swift at 10