How to Safely Transfer Data or Verify Backups
Suppose that you are switching to a new system. Or that you have upgraded to a newer and larger hard drive. Or swapped SSDs. Or that you burned files to a DVD or BluRay and are they any good a year or two later. Etcetera.
That’s why I wrote IntegrityChecker, part of diglloydTools: to avoid “bit rot” and corruption from transfers. It removes the worry from the process.
Using a cryptographic hash (SHA1) IntegrityChecker makes a record in each folder for each file in that folder of its hash. This saved hash detects the tiniest change in content (or of course, length of the file too).
The type of file does not matter: image files, Lightroom catalogs, JPEGs, raw files, spreadsheets, mail, word processing files, music, videos, databases, etc—anything that you create and has value to you.
The process is simple and can be run on a single folder or entire volume.
- Run (computes and writes the hash values for every file in each folder into a hidden “.ic” file in that folder). on the original files
- Make the copy or backup or burn the DVD/Blu Ray or whatever (this naturally carries along the hidden “.ic” file in each folder).
- At any later time (tomorrow or a year later), run (this recomputes the hashes and compares to the values in the “.ic” file). on the backup / copy
IntegrityChecker can verify any files on which 'Update' run before, even without the originals (there are alternatives such as “diffing” source and backup, but those are far slower, and require the original and the copy).
Some pro photographers burn DVD or BluRay discs containing files with IntegrityChecker support; these discs can be verified at any time. There are numerous such uses.
See How to Safely Transfer Data or Verify Backups and also Example of Verifying Data Integrity.
Both command line (Terminal) and GUI versions are provided. The GUI is not fancy, but the internals are what counts: one of the most efficient multi-threaded programs of any kind you’ll ever find.