All Posts by Date or last 15, 30, 90 or 180 days.
also by Lloyd: photography and

Links on this site earn me fees or commissions.
As an Amazon Associate I earn from qualifying purchases @AMAZON

Consult with Lloyd: cameras, computers, backup, etc...
Lloyd’s Patreon
Designed for the most demanding needs of photographers and videographers.
The fastest, toughest, and most compatible portable SSD ever with speeds up to 2800MB/s.

IntegrityChecker Java: Simultaneous Data Verification on any Number of Hard Drives or SSDs

See also: Checking for Data Corruption after macOS Hoses Your System

I let some of my tertiary backup drives go stale for some months, and finally brought them all up to day today. I like to do that after coming back from a trip.

With hard drives, multiple backups can take a loooooong time, with the 8TB drives and 14TB drives mostly full, though there are some tricks to also doing simultaneous backups*.

After backing up, verifying data integrity on all 8 drives can be done very efficiently with IntegrityChecker Java (icj). All eight drives can be verified simultaneously*, using one instance of icj (one per Terminal window).

Since each drive is independent of the others, the only gating factors are I/O bandwidth (no problem in an OWC Thunderbay), and CPU power (ample on the 8-core 2018 iMac 5K. The hard drives are slow enough that each drive takes only ~35% of one CPU core). In other words, the time it takes depends entirely on how fast (slow) the drives are. A 4-core CPU would see higher CPU utilization of course.

Avoid simultaneous backups or data verification if the data is on the same physical hard drive (such as two volume partitions); it will cause the drive to bang its drive heads continuously and run very slower. SSDs do not have this limitation, as there are no moving parts.

IntegrityChecker Java: Simultaneous Data Verification on any Number of Hard Drives at Once

Below, 8 backup volumes totaling 71TB were validated, showing perfect data integrity—not a single bit in error.

IntegrityChecker Java: 8 backup volumes totaling 71TB validated with perfect data integrity
View all handpicked deals...

Sony X80K 75" 4K HDR Smart LED TV
$1198 $1098
SAVE $100 | Terms of Use | PRIVACY POLICY
Contact | About Lloyd Chambers | Consulting | Photo Tours
Mailing Lists | RSS Feeds | Twitter
Copyright © 2020 diglloyd Inc, all rights reserved.
Display info: __RETINA_INFO_STATUS__