Are Your Backup Drives Still Functional? Are Your Files Subtly Damaged?
diglloydTools includes IntegrityChecker, which can verify file integrity at any later date* whether on a hard drive, SSD, DVD, BluRay or network volume.
The verification does require that the files have been previously hashed*, but once done this allows verification at any time on any media.
IntegrityChecker is a workhorse tool that I developed to ensure the integrity of own data. Backup is not enough by itself if the drive is developing marginal or bad sectors; you cannot necessarily know if the files are intact or not. And in the case of a restore, it’s really nice to know that the files are intact; simply being able to copy them is no proof that the contents are undamaged.
Thus it is good backup hygiene to run IntegrityChecker 'verify' on your backup drives every 4-6 months, to make sure that new issues have not developed. Especially with cloning, files that are unchanged are not rewritten, and it is possible for files to go bad as the drive media is not always 100% reliable**.
* A hidden “.ic” file is stored in each folder containing SHA-1 hashes of all files in that folder. Moving or copying or backing-up a folder thus naturally carries along these file hashes.
** An erase and re-backup cycle is advised once a year in order to write fresh copies onto hard drives, just in case there are weak sectors involved; rewriting refreshes things magnetically. For the same reasons, discard drives after 4 years or so; it’s not worth it.
Sometimes it’s subtle and that is where IntegrityChecker is valuable.
- How to Safely Transfer Data or Verify Backups
- Example of Verifying Data Integrity.
- How to Automate DiskUtility Verification of Mounted Volumes.
Drives go bad for no obvious reason
Sometimes drive failure is not subtle (preferred): I just trash-canned four 2TB Western Digital RE3 enterprise-grade drives—all of them had gone bad for no reason I could discern. And those are the enterprise grade drives, which are allegedly better.
Steve D writes:
1) Do you recommend Integrity Checker update - verify on a Time Machine backup volume?
2) Do you recommend Integrity Checker on /System?
Many thanks for your great software!
On (1), since TM backs up all files including the ".ic" file, yes the Time Machine backup volume can be scanned like any other file system. It is worthwhile as a sanity check if you suspect problems on the TM backup volume. However, scanning the entire volume will take 'forever' due to the way Time Machine stores its snapshots; you’ll want to selectively do 'ic verify' only on the 'Latest' folder.
It is a good idea to erase the Time Machine volume every six months* or so to save space (do you need the last 23 copies of a frequently edited file?). By erasing the TM drive, you force a new snapshot which rewrites all files and eliminates all history.
Keep other backups especially clone backups, and bring these up to date before erasing the TM volume. TM is best used for short term protection, not a first-level backup since it is subject to physical risks the same as the computer.
On (2), IntegrityChecker can be used on the system volume at the command line using 'sudo', but in general this is best used for system transfer (do 'ic clean' after an 'ic verify'). The issue is that various application bundles can report as modified due to the ".ic" file. The cleanest approach is to make a disk image clone, then use IntegrityChecker on that disk image, which of course presumes no errors on the volume containing that disk image. So the only sure way is to