All Posts by Date or last 15, 30, 90 or 180 days.
also by Lloyd: diglloyd.com photography and WindInMyFace.com
Thank you for purchasing through links and ads on this site.
OWC / MacSales.com...
diglloyd B&H Deal Finder...
Buy other stuff at Amazon.com...
Up to 1527MB/s sustained performance
877-865-7002
Today’s Deal Zone Items... Handpicked deals...
$1499 $1049
SAVE $450

$1699 $1199
SAVE $500

$799 $299
SAVE $500

$15199 $13699
SAVE $1500

$399 $399
SAVE $click

$7995 $7495
SAVE $500

$1399 $1049
SAVE $350

$450 $450
SAVE $click

$1299 $1199
SAVE $100

$700 $500
SAVE $200

$764 $399
SAVE $365

$290 $250
SAVE $40

$169 $99
SAVE $70

$370 $320
SAVE $50

$748 $648
SAVE $100

$200 $170
SAVE $30

$360 $360
SAVE $click

$147 $80
SAVE $67

$1648 $1278
SAVE $370

$250 $235
SAVE $15

$3899 $3899
SAVE $click

$2799 $2499
SAVE $300

$1199 $999
SAVE $200

$5699 $4699
SAVE $1000

$2999 $2399
SAVE $600

$18599 $17099
SAVE $1500

$600 $600
SAVE $click

$1149 $799
SAVE $350

$150 $50
SAVE $100

$420 $170
SAVE $250

$1597 $1097
SAVE $500

$1499 $679
SAVE $820

$1499 $679
SAVE $820

$420 $170
SAVE $250

$369 $228
SAVE $141

$1699 $1399
SAVE $300

$1498 $998
SAVE $500

$999 $949
SAVE $50

$1099 $999
SAVE $100

OWC Envoy Pro EX SSD
Blazingly fast Thunderbolt 3 SSD!

Up to 4TB capacity, USB-C compatible.

USB-C model also available


Great for travel or for desktop!

Validating Data Transfer/Copy Before Deleting the Source/Original Data: diglloydTools IntegrityChecker

See also Data Integrity Over Time, and with OS Changes and Data Loss Prevented: IntegrityChecker Saves my Bacon by Detecting Corrupted Files after a Clone.

For background see: Detecting Corruption / Validating Data Integrity Over Time and Across Drives and Backup/Restore.

Every time I take a photography trip, I come back with something between 100GB and 800GB of new image files. These files typically reside on the internal SSD of my 2019 iMac 5K, which I use in the field in my Mercedes Sprinter photography adventure van.

When I return home, the first thing I do is to offload the trip photos to my primary working drive (an 8TB OWC Thunderblade). Recent years have shown me that the macOS Finder is prone to some serious bugs with file copying, so there is just no way I am going to assume that it did the copy correctly.

Accordingly, I want to verify that the transferred items are absolutely identical before I delete the items off the internal SSD.

Procedure for validating a data transfer

Here is my procedure, which presumes a backup has already been made:

  1. Run IntegrityChecker 'update' on the files to be copied*.
  2. Copy the files to their new home.
  3. Run IntegrityChecker verify on the copies and the originals*.
  4. Compare the results—they should be identical with no warnings of any kind.
  5. Assuming things are identical, delete the originals.

This procedure guarantees that no matter what might happen, any issue in the copying process (missing/skipped files, bit flips, etc) are caught. Or for that matter, anything else nefarious that might possibly happen.

* Use the Java version, as it can also flag folders that have been deleted since the last update (a missing folder by definition has no files and no hashes).

Example

On the source drive (internal SSD):

Do an 'update' first as noted above. The verify will then show "Without hash: 0", because all files will have a cryptographic hash.

The elegance of this approach is terrific: at any future time, this copy or the 100th copy of this copy on any media can be verified again—no original needed, just verify the copy. A single command verifies the folder and all subfolders, whether 1MB or 500 terabytes.

diglloyd-iMac:2019-1204-TripPhotos lloyd$ icj verify /Master/2019-1204-TripPhotos/
...
=================================================================================
FILE STATUS SUMMARY for 105 folders 2019-12-18 14:48:34
=================================================================================
# With hash: 14029
# Without hash: 0
# Missing : 0
# Hashed: 14029
# Changed size: 0
# Changed date: 0
# Changed content + date, size unchanged: 0
# Total changed content: 0
# SUSPICIOUS: 0

On the destination drive (external SSD):

The files have been copied. Verify them. The number of files "With hash" should match and there should be zero files with changes of any kind.

diglloyd-iMac:DIGLLOYD lloyd$ icj verify /Work/2019-1204-TripPhotos/
.....
==================================================================================
FILE STATUS SUMMARY for 105 folders 2019-12-18 16:12:23
==================================================================================
# With hash: 14029
# Without hash: 0
# Missing : 0
# Hashed: 14029
# Changed size: 0
# Changed date: 0
# Changed content + date, size unchanged: 0
# Total changed content: 0
# SUSPICIOUS: 0
OWC Accelsior 4M2 PCIe SSD
6000 MB/sec!
Mac or PC.


Ideal for Lightroom, Photoshop, video.
Capacity up to 16TB!
Save Big $$$$ on Memory for 2019 Mac Pro

Up to 65% better pricing than Apple

Lloyd recommends 32GB RDIMM modules for most users (more expensive LRDIMMS are for 512GB or more).



Save the tax, we pay you back, instantly!
View all handpicked deals...

Apple iPad Pro 12.9" (512GB, Wi-Fi + 4G LTE, Space Gray, Previous Gen)
$1499 $1049
SAVE $450

diglloyd.com | Terms of Use | PRIVACY POLICY
Contact | About Lloyd Chambers | Consulting | Photo Tours
Mailing Lists | RSS Feeds | Twitter
Copyright © 2020 diglloyd Inc, all rights reserved.
Display info: __RETINA_INFO_STATUS__