SEARCH
diglloydTools™
ZFS for Mac OS X — ZEVO
As drives become huge, the risk of data corruption becomes an ever increasing concern. Corruption can be subtle— “bit rot”, bit flips during file transfers, etc, and these can accrete over time.
The ZFS file system (developed at Sun but open source at OpenSolaris.org) addresses almost all the nasties which can destroy user data— which can be an invisible and silent risk until those files are needed— not just a drive crash or more obvious symptoms.
Some criticism of Apple here— Apple is intently focused on improving the user experience. But how one creates a quality user experience depends on how one defines it; it is not a quality user experience when the last 5 years of one’s computing life vaporize. Apple has made some steps in that direction (Time Machine for example), but has not addressed the fundamental weaknesses in the file system.
A quality user experience presumes a reliable computing platform , and reliable storage for data. And I’m not talking about mail and contacts and reminders and games and to-do notes, but about photos, video, etc— stuff that cannot be regenerated.
Apple’s HFS+ file system is an aging design that deserves to be retired. That’s where ZFS comes in, and it would be ideal if Apple finally moved to ZFS as the default file system in OS X 'Mountain Lion'. But the focus is elsewhere, on iOS, for which one can hardly blame Apple, given the huge profits involved. Yet the changes in OS X Mountain Lion are almost all application-level changes, not credible operating system improvements.
ZEVO — ZFS for Mac OS X
Technically savvy users might want to check out ZEVO (thanks to the note from Stan H below). It is regrettable that Apple has not taken the lead in making ZFS the default file system on OS X, but at least there is an option.
I have not looked into ZEVO myself, and I probably should. But I also maintain a robust backup strategy as well as my IntegrityChecker program, and to a real extent I need to stay mainstream because of this site. But professionals whose data is their livelihood should at least take a look at ZEVO, and should at least read the ZFS presentation to understand the issues involved in data integrity.
Stan H writes:
You might want to make a little announcement about the recent availability of ZFS for OS X called ZEVO. I am sure you have heard of it. It is a commercial version of ZFS from a company called Ten's Complement started by Don Brady, an ex long-time engineer at Apple who was on the early HFS team and the discontinued internal ZFS project at Apple.
I have been testing the initial release for the last two weeks and it works beautifully. While the initial release has limited functionality, it provides the core protection of complete check-summing of all data, redundancy of metadata and, if optionally enabled, same disk redundancy of data.
ZEVO has a nice simple GUI and integrates quite seamlessly into OS X. Mirroring, RAIDZ and all the other sophisticated stuff is coming soon but for $20 for the initial release I want my data protected now. A must have for anyone, especially photographers, with irreplaceable data.
Tore O writes:
Thank you for a very useful site. I have significantly improved my Macintosh experience by learning from your articles. I have a few comments on the Zevo/ZFS software you just wrote about.
Recently, in the course of two days, I experienced directory corruption on 5 hard drives, including my 4TB Time machine disk. This was due to a stuck address in a RAM block. Thankfully I use Crashplan to backup to a ZFS based OpenSolaris server, so my data were easily restored after replacing the bad RAM block. Granted, RAM errors are presumably rare occurrences and this might never have happened on a Mac Pro which has ECC RAM, but it could happen to anyone else using an iMac or other non-professional Mac. Let us hope Apple does not drop the Mac Pro.
After this incidence I was quite motivated to find a better filesystem for my precious data on the Mac. So I started beta-testing Z410/Zevo and we now own the commercial version. Zevo does a beautiful job of making ZFS transparent with no command line use needed for basic ZFS protection. However, while the developer has brought to market a great product, Ten's Complement is still working on optimizing performance. So if you take it for a spin, know that there are speed improvements coming. Also a few important features are not implemented yet. These include encryption, enabling auto replacing of a failed disk with a hot spare, and booting from ZFS disks.
One caveat to anyone wanting to get started with Zevo: The product that you mention (Zevo Silver edition) offers metadata redundancy which is already a big step forward relative to HFS+, but file redundancy requires you to go to the command line and set a "copies=2" option which will slow down writes significantly (since the drive needs to write the file twice to the same disk) and still will not protect against drive failure.
Photographers and others with critical data will want to use either Zevo Gold or Zevo Platinum edition which should be available later this year and will provide drive-failure tolerant storage. I would be very interested to hear your thoughts if you were to test the Zevo Gold or Platinum edition when they become available. In particular, ZFS has the exciting prospect of allowing you to attach solid state drives as read-cache and write log devices to hard drive based storage pools. This should speed up reads and asynchronous writes. If you test Zevo at some point, it would be very interesting to hear your observations on if and how this would speed up/improve the user experience in database driven applications like Adobe Lightroom or Apple Aperture.
Regards,
TorePS: I find your DiskTester product very useful for testing hard drive performance and have successfully reconditioned old SSDs with it. Thank you very much.
DIGLLOYD: Indeed, ECC memory is a Big Deal now that memory can be up to 96GB in a 12-core Mac Pro (more memory = great chance of a bit flip somewhere). Laptops are at far greater risk because users tend to leave them in sleep mode on airplanes, where the chance of a cosmic ray flipping bits is much higher at altitude.
Directory corruption is the worst possible case, since it can affect everything on a drive. That is bad luck indeed.
I will likely test ZFS / ZEVO at some point, but I think I will let it mature.
As for SSD caching, that's a worthwhile idea, but more complex (complexity has to be weighed against real benefits), and wholly irrelevant to my work: I just always work off the SSDs anyway, migrating older material to hard drives.
