All Posts by Date or last 15, 30, 90 or 180 days.
also by Lloyd: diglloyd.com photography and WindInMyFace.com

Thank you for buying via links and ads on this site,
which earn me advertising fees or commissions.
As an Amazon Associate I earn from qualifying purchases.

Other World Computing...
B&H Photo...
Amazon
As an Amazon Associate I earn from qualifying purchases.
Capacities up to 56TB and speeds up to 1527MB/s
877-865-7002
Today’s Deal Zone Items... Handpicked deals...
$1199 $779
SAVE $420

$249 $219
SAVE $30

$1699 $1299
SAVE $400

$1499 $1149
SAVE $350

$200 $140
SAVE $60

$89 $69
SAVE $20

$120 $35
SAVE $85

$209 $179
SAVE $30

$3498 $3498
SAVE $click

$3699 $1299
SAVE $2400

$1000 $700
SAVE $300

$3899 $1499
SAVE $2400

$748 $648
SAVE $100

$2999 $1999
SAVE $1000

$1699 $999
SAVE $700

$1698 $1398
SAVE $300

$4499 $3499
SAVE $1000

$1399 $1049
SAVE $350

$1149 $799
SAVE $350

$1399 $1049
SAVE $350

$1498 $1198
SAVE $300

$2797 $2597
SAVE $200

$280 $250
SAVE $30

$700 $500
SAVE $200

$1699 $1299
SAVE $400

$8399 $7599
SAVE $800

$899 $799
SAVE $100

$1699 $1299
SAVE $400

OWC Envoy Pro EX SSD
Blazingly fast Thunderbolt 3 SSD!

Up to 4TB capacity, USB-C compatible.

USB-C model also available


Great for travel or for desktop!

Validating Data Transfer/Copy Before Deleting the Source/Original Data: diglloydTools IntegrityChecker

See also Data Integrity Over Time, and with OS Changes and Data Loss Prevented: IntegrityChecker Saves my Bacon by Detecting Corrupted Files after a Clone.

For background see: Detecting Corruption / Validating Data Integrity Over Time and Across Drives and Backup/Restore.

Every time I take a photography trip, I come back with something between 100GB and 800GB of new image files. These files typically reside on the internal SSD of my 2019 iMac 5K, which I use in the field in my Mercedes Sprinter photography adventure van.

When I return home, the first thing I do is to offload the trip photos to my primary working drive (an 8TB OWC Thunderblade). Recent years have shown me that the macOS Finder is prone to some serious bugs with file copying, so there is just no way I am going to assume that it did the copy correctly.

Accordingly, I want to verify that the transferred items are absolutely identical before I delete the items off the internal SSD.

Procedure for validating a data transfer

Here is my procedure, which presumes a backup has already been made:

  1. Run IntegrityChecker 'update' on the files to be copied*.
  2. Copy the files to their new home.
  3. Run IntegrityChecker verify on the copies and the originals*.
  4. Compare the results—they should be identical with no warnings of any kind.
  5. Assuming things are identical, delete the originals.

This procedure guarantees that no matter what might happen, any issue in the copying process (missing/skipped files, bit flips, etc) are caught. Or for that matter, anything else nefarious that might possibly happen.

* Use the Java version, as it can also flag folders that have been deleted since the last update (a missing folder by definition has no files and no hashes).

Example

On the source drive (internal SSD):

Do an 'update' first as noted above. The verify will then show "Without hash: 0", because all files will have a cryptographic hash.

The elegance of this approach is terrific: at any future time, this copy or the 100th copy of this copy on any media can be verified again—no original needed, just verify the copy. A single command verifies the folder and all subfolders, whether 1MB or 500 terabytes.

diglloyd-iMac:2019-1204-TripPhotos lloyd$ icj verify /Master/2019-1204-TripPhotos/
...
=================================================================================
FILE STATUS SUMMARY for 105 folders 2019-12-18 14:48:34
=================================================================================
# With hash: 14029
# Without hash: 0
# Missing : 0
# Hashed: 14029
# Changed size: 0
# Changed date: 0
# Changed content + date, size unchanged: 0
# Total changed content: 0
# SUSPICIOUS: 0

On the destination drive (external SSD):

The files have been copied. Verify them. The number of files "With hash" should match and there should be zero files with changes of any kind.

diglloyd-iMac:DIGLLOYD lloyd$ icj verify /Work/2019-1204-TripPhotos/
.....
==================================================================================
FILE STATUS SUMMARY for 105 folders 2019-12-18 16:12:23
==================================================================================
# With hash: 14029
# Without hash: 0
# Missing : 0
# Hashed: 14029
# Changed size: 0
# Changed date: 0
# Changed content + date, size unchanged: 0
# Total changed content: 0
# SUSPICIOUS: 0
OWC Thunderblade Thunderbolt 3 SSD

Blazing fast, up to 16TB.

YEE HAH!



√ No more slow and noisy hard drives!
OWC Accelsior 4M2 PCIe SSD
6000 MB/sec!
Mac or PC.


Ideal for Lightroom, Photoshop, video.
Capacity up to 16TB!
OWC Envoy Express

World’s first Thunderbolt 3 M.2 NVME SSD enclosure.

Super fast, bus-powered, 3.3 oz, DIY easy, rugged and compact!


See also OWC Express 4M2
√ No more slow and noisy hard drives!

MacPerformanceGuide.com
View all handpicked deals...

LG UM7370PUA 70" Class HDR 4K UHD Smart LED TV
$1199 $779
SAVE $420

diglloyd.com | Terms of Use | PRIVACY POLICY
Contact | About Lloyd Chambers | Consulting | Photo Tours
Mailing Lists | RSS Feeds | Twitter
Copyright © 2020 diglloyd Inc, all rights reserved.
Display info: __RETINA_INFO_STATUS__