All Posts by Date or last 15, 30, 90 or 180 days.
also by Lloyd: photography and

Links on this site earn me fees or commissions.
As an Amazon Associate I earn from qualifying purchases @AMAZON

Consult with Lloyd: cameras, computers, backup, etc...
Lloyd’s Patreon
Designed for the most demanding needs of photographers and videographers.

Are Your Backup Drives Still Functional? Are Your Files Subtly Damaged?


diglloydTools includes IntegrityChecker, which can verify file integrity at any later date* whether on a hard drive, SSD, DVD, BluRay or network volume.

The verification does require that the files have been previously hashed*, but once done this allows verification at any time on any media.

IntegrityChecker is a workhorse tool that I developed to ensure the integrity of own data. Backup is not enough by itself if the drive is developing marginal or bad sectors; you cannot necessarily know if the files are intact or not. And in the case of a restore, it’s really nice to know that the files are intact; simply being able to copy them is no proof that the contents are undamaged.

Thus it is good backup hygiene to run IntegrityChecker 'verify' on your backup drives every 4-6 months, to make sure that new issues have not developed. Especially with cloning, files that are unchanged are not rewritten, and it is possible for files to go bad as the drive media is not always 100% reliable**.

* A hidden “.ic” file is stored in each folder containing SHA-1 hashes of all files in that folder. Moving or copying or backing-up a folder thus naturally carries along these file hashes.

** An erase and re-backup cycle is advised once a year in order to write fresh copies onto hard drives, just in case there are weak sectors involved; rewriting refreshes things magnetically. For the same reasons, discard drives after 4 years or so; it’s not worth it.

Sometimes it’s subtle and that is where IntegrityChecker is valuable.

See also:

Drives go bad for no obvious reason

Sometimes drive failure is not subtle (preferred): I just trash-canned four 2TB Western Digital RE3 enterprise-grade drives—all of them had gone bad for no reason I could discern. And those are the enterprise grade drives, which are allegedly better.

Steve D writes:

1) Do you recommend Integrity Checker update - verify on a Time Machine backup volume?

2) Do you recommend Integrity Checker on /System?

Many thanks for your great software!


On (1), since TM backs up all files including the ".ic" file, yes the Time Machine backup volume can be scanned like any other file system. It is worthwhile as a sanity check if you suspect problems on the TM backup volume. However, scanning the entire volume will take 'forever' due to the way Time Machine stores its snapshots; you’ll want to selectively do 'ic verify' only on the 'Latest' folder.

It is a good idea to erase the Time Machine volume every six months* or so to save space (do you need the last 23 copies of a frequently edited file?). By erasing the TM drive, you force a new snapshot which rewrites all files and eliminates all history.

Keep other backups especially clone backups, and bring these up to date before erasing the TM volume. TM is best used for short term protection, not a first-level backup since it is subject to physical risks the same as the computer.

On (2), IntegrityChecker can be used on the system volume at the command line using 'sudo', but in general this is best used for system transfer (do 'ic clean' after an 'ic verify'). The issue is that various application bundles can report as modified due to the ".ic" file. The cleanest approach is to make a disk image clone, then use IntegrityChecker on that disk image, which of course presumes no errors on the volume containing that disk image. So the only sure way is to

View all handpicked deals...

Nanuk 925 Waterproof Hard Case Pro Photo/Video Kit with Padded Dividers and Lid Organizer (Black)
$285 $214
SAVE $71 | Terms of Use | PRIVACY POLICY
Contact | About Lloyd Chambers | Consulting | Photo Tours
Mailing Lists | RSS Feeds |
Copyright © 2020 diglloyd Inc, all rights reserved.
Display info: __RETINA_INFO_STATUS__