Does macOS Have a Rare but Stupefyingly Nasty Multithreading Bug in the Unified Buffer Cache?

2020-01-30 • SEND FEEDBACK |
Related: Apple, Apple Core Rot, Apple macOS, BBEdit, caching, multithreading

macOS caches files up the wazoo, using all available memory to do so. On my system with 384GB memory, I have seen it cache nearly 350GB, an idiotic performance bug that degrades everything else with the system.

With 350GB cached, it can take up to 90 seconds to clear the cache (!!!), with program performance being hurt everywhere else as the system pauses to reclaim the cache. You can see this as apps stutter or are delayed at launch or any similar circumstances. The greater the number of files in the cache, the greater the performance hit, e.g., 250K files occupying 350GB hits it hard.

Performance is one thing but incorrect operation is the kiss of death

I hope I have just repeatedly had hallucinations and/or that there is a perfectly good explanation I have not thought of. But I didn’t write multithreaded code for 30 years as a professional software engineer to be easily fooled.

I’ve seen the following behavior with several programs now and I’ve seen it several times in several months. This bug is SO BAD with potential consequences so severe that I thought I must have imagined it when I had seen it before.

It’s a simple behavior, but I don’t know what makes for a reproducible test case—it happens when it happens but what triggers it is a mystery:

1. Make use of a file in Program A. For example, run a script in Terminal.
2. Save a CHANGE to the file in Program B, such as BBEdit, Dreamweaver, etc.
3. Make use of the file again.
4. Observe that Program A sees the OLD FILE, the one before the save.

Two days ago, I watched a script fail to update as I repeatedly invoked it and repeatedly hit Save in BBEdit to confirm I had saved the file. That it was wrong was obvious: an output folder name repeated a few hundred times. NO CHANGE as seen by Terminal as I repeatedly invoked the script and saw it fail over and over with the wrong folder name, the name having been changed in BBEdit. I switched back and forth several times to save and try again—NO CHANGE. The file was apparently cached somewhere (presumably by macOS though I cannot be certain).

The seriousness of a bug like this cannot be overestimated. It bears on data destruction for starters, but also could be a serious security hole.

I am having the same problem EVERY DAY saving files in DreamWeaver and then having the new page not show up for a second or two or half a minute; I often have to make a change and save again or again... 2 or 3 or even 4 times.

The frequency of this increased with macOS Catalina, but I saw it in Mojave too.

I suspect that it is a multithreading bug since I now have 16 CPU cores in the 2019 Mac Pro (28 cores starting today). Therefore, any threading bugs are likely to be far worse with 16 cores versus 8 as on the 2019 iMac 5K.

Paul W writes:

I've seen this bug many times since 10.12 at least. Frustratingly annoying. Sometimes I reboot, sometimes I wait until I start seeing the "correct" copy. Sometimes I copy the file to a new file and work with that. I've never bothered to try to diagnose it further. Also, I spent lots of time at the command line.

DIGLLOYD: I keep wondering if something else is involved than the unified buffer cache, but I don't quite see how.